diff --git a/docs/modules/automation-testing/pages/scripting/auto-generate-an-appium-script.adoc b/docs/modules/automation-testing/pages/scripting/auto-generate-an-appium-script.adoc index dc35751b0..6f4f564c6 100644 --- a/docs/modules/automation-testing/pages/scripting/auto-generate-an-appium-script.adoc +++ b/docs/modules/automation-testing/pages/scripting/auto-generate-an-appium-script.adoc @@ -5,7 +5,7 @@ Learn how to generate an Appium script from a manual session, so you can start b == Before you start -* xref:scriptless-automation:create-a-baseline-session.adoc[] +* xref:_scriptless-automation:create-a-baseline-session.adoc[] [#_export_your_script] == Export your script @@ -14,7 +14,7 @@ After creating the baseline session, search for your session, open the session o image:automation-testing:export-appium-script-context.png[width=1200, alt="The automated test case for the selected session."] -Next, select the *Export Appium Script* icon. If you don't see an icon, check if your session xref:test-management:remediation/index.adoc[requires remediation]. +Next, select the *Export Appium Script* icon. If you don't see an icon, check if your session xref:_test-management:remediation/index.adoc[requires remediation]. image:automation-testing:export-appium-script-closeup.png[width=750, alt="Session overview tabs, including Automated Test Case, Jira Integration, Rerun, and the Appium Export icon."] diff --git a/docs/modules/organization/pages/device-bundles/create-a-device-bundle.adoc b/docs/modules/organization/pages/device-bundles/create-a-device-bundle.adoc index 7e6f9d526..bcff5bb54 100644 --- a/docs/modules/organization/pages/device-bundles/create-a-device-bundle.adoc +++ b/docs/modules/organization/pages/device-bundles/create-a-device-bundle.adoc @@ -1,7 +1,7 @@ = Create a device bundle :navtitle: Create a device bundle -Learn how to create a new device bundle so users in your organization can run xref:scriptless-automation:index.adoc[Scriptless Automation] on devices in parallel. +Learn how to create a new device bundle so users in your organization can run xref:_scriptless-automation:index.adoc[Scriptless Automation] on devices in parallel. [NOTE] Device bundles are operating system (OS) specific so Android and iOS devices cannot be bundled together. diff --git a/docs/modules/organization/pages/device-bundles/manage-device-bundles.adoc b/docs/modules/organization/pages/device-bundles/manage-device-bundles.adoc index 7fc90802e..be0e64e18 100644 --- a/docs/modules/organization/pages/device-bundles/manage-device-bundles.adoc +++ b/docs/modules/organization/pages/device-bundles/manage-device-bundles.adoc @@ -1,7 +1,7 @@ = Manage device bundles :navtitle: Manage device bundles -Learn how to manage the device bundles in your organization so users can use them to run xref:scriptless-automation:index.adoc[Scriptless Automation] on devices in parallel. You can only manage device bundles xref:device-bundles/create-a-device-bundle.adoc[you or your team created]. +Learn how to manage the device bundles in your organization so users can use them to run xref:_scriptless-automation:index.adoc[Scriptless Automation] on devices in parallel. You can only manage device bundles xref:device-bundles/create-a-device-bundle.adoc[you or your team created]. == View a device bundle diff --git a/docs/modules/reporting/pages/usage-report/report-metadata.adoc b/docs/modules/reporting/pages/usage-report/report-metadata.adoc index 4f1e7df3a..08b866162 100644 --- a/docs/modules/reporting/pages/usage-report/report-metadata.adoc +++ b/docs/modules/reporting/pages/usage-report/report-metadata.adoc @@ -44,23 +44,23 @@ xref:manual-testing:start-a-mixed-session.adoc[Mixed test sessions]. === Scriptless on Manual -xref:scriptless-automation:index.adoc[Scriptless test sessions] that use a manual test session as the base. +xref:_scriptless-automation:index.adoc[Scriptless test sessions] that use a manual test session as the base. === Scriptless on Automation -xref:scriptless-automation:index.adoc[Scriptless test sessions] that use an automation test session as the base. +xref:_scriptless-automation:index.adoc[Scriptless test sessions] that use an automation test session as the base. === Test Case -xref:test-management:index.adoc[Test cases]. +xref:_test-management:index.adoc[Test cases]. === Text Validation -xref:test-management:validation/text-validation.adoc[Text validation] within a Scriptless session. +xref:_test-management:validation/text-validation.adoc[Text validation] within a Scriptless session. === Color Validation -xref:test-management:validation/color-text-validation.adoc[Color text validation] within a Scriptless session. +xref:_test-management:validation/color-text-validation.adoc[Color text validation] within a Scriptless session. === Font Size (WBI) diff --git a/docs/modules/scriptless-automation/images/capture-session-icon.png b/docs/modules/scriptless-automation/images/capture-session-icon.png new file mode 100644 index 000000000..7fd8cd6a3 Binary files /dev/null and b/docs/modules/scriptless-automation/images/capture-session-icon.png differ diff --git a/docs/modules/scriptless-automation/images/convert-test-case.png b/docs/modules/scriptless-automation/images/convert-test-case.png new file mode 100644 index 000000000..a39a04cf7 Binary files /dev/null and b/docs/modules/scriptless-automation/images/convert-test-case.png differ diff --git a/docs/modules/scriptless-automation/nav.adoc b/docs/modules/scriptless-automation/nav.adoc index 74637104c..80e4f8d7a 100644 --- a/docs/modules/scriptless-automation/nav.adoc +++ b/docs/modules/scriptless-automation/nav.adoc @@ -1,13 +1,6 @@ .xref:index.adoc[] -* xref:scriptless-automation:create-a-baseline-session.adoc[] - -* Get a session ID -** xref:get-a-session-id/using-the-kobiton-portal.adoc[] -** xref:get-a-session-id/using-the-kobiton-api.adoc[] - -* xref:run-scriptless-with-the-kobiton-portal.adoc[] -* xref:run-scriptless-with-the-kobiton-api.adoc[] -* xref:automation-testing:scripting/auto-generate-an-appium-script.adoc[] -* xref:scriptless-best-practices.adoc[] -* xref:scriptless-requirements.adoc[] +* xref:scriptless-automation:baseline-session.adoc[] +* xref:scriptless-automation:run-scriptless.adoc[] +* xref:scriptless-automation:run-scriptless-api.adoc[] +* xref:scriptless-automation:best-practices.adoc[] diff --git a/docs/modules/scriptless-automation/pages/baseline-session.adoc b/docs/modules/scriptless-automation/pages/baseline-session.adoc new file mode 100644 index 000000000..d26530ea9 --- /dev/null +++ b/docs/modules/scriptless-automation/pages/baseline-session.adoc @@ -0,0 +1,80 @@ += Create a baseline session + +== Overview + +A baseline session is a manual test session that serves as the source for automated test execution. It captures how an application is exercised on a real device and provides the reference used to create a test case. + +Baseline sessions are created through manual interaction in the Kobiton platform and are required before a test can be run automatically. + +== What is a baseline session? + +A baseline session is a manual session performed on a real device through the platform. + +During the session, a user interacts with the application as they would during normal testing. These interactions are captured by the platform and later reused to define a test flow. + +Once converted into a test case, the baseline session becomes the reference for future test execution. Test cases can also be used to generate Appium scripts. + +== Create a baseline session + +Baseline sessions are created by running a manual test session with capture enabled and completing a defined test flow. + +=== 1. Start a manual session and enable capture + +From the Devices view in the Kobiton platform, select a device and launch a manual session. + +Before interacting with the application, select the *Play* button to enable session capturing mode. + +image::capture-session-icon.png[width=600,alt="Capture Session icon in Session Explorer"] + +Session capturing mode improves the accuracy of element detection by capturing precise element bounding boxes. Actions performed while capturing is enabled provide the most reliable results for scriptless execution. + +If your test requires an application, install or select the app at the start of the session. + +NOTE: Actions performed before capture is enabled are still saved with the session, but may result in reduced element accuracy during automated execution. + +=== 2. Perform the test flow + +Interact with the application as you would during normal testing. + +While capture is enabled, the platform captures supported interactions and associates them with the current application state. These captured interactions form the basis of the baseline session. + +Focus on completing a single, intentional test flow from start to finish. + +=== 3. End the session and convert it to a test case + +When the test flow is complete, exit the manual session by selecting the *X* icon. + +The session then opens in Session Explorer, where it can be converted into a test case. Once converted, the test case is managed through Test Management, where it can be reviewed, edited, and used to create test runs. + +To convert a session into a test case or learn more about Test Management, see xref:test-management:test-cases.adoc[Test cases]. + +image::convert-test-case.png[width=600,alt="Convert Test Case in Session Explorer"] + +== What makes a good baseline + +Baseline sessions are most reliable when they reflect a clear and stable test flow. + +Good baseline sessions typically: + +* Follow a single, intentional path through the application +* Use supported interactions only +* Avoid unnecessary navigation or exploratory actions + +Baseline quality directly affects how reliably a test can be executed later. + +== Platform considerations + +Baseline sessions are platform-specific. + +Separate baseline sessions are required for Android and iOS, even when the application behavior is similar. + +Baseline sessions must target a supported application context: a web application (Safari or Chrome), an app installed from the App Repository, or an existing app already available on the device. + +Manual sessions can be converted into test cases even if unsupported actions were captured. Test cases that contain unsupported actions cannot be executed in a test run until those steps are removed and the test case is saved as a new version. + +== Related pages + +* xref:test-management:test-cases.adoc#_convert_to_test_case[Convert a session to a Test Case] +* xref:scriptless-automation:run-scriptless.adoc[Run a Scriptless test] +* xref:test-management:index.adoc[Test management] +* xref:scriptless-automation:best-practices.adoc[Best practices] diff --git a/docs/modules/scriptless-automation/pages/best-practices.adoc b/docs/modules/scriptless-automation/pages/best-practices.adoc new file mode 100644 index 000000000..fbe94a9c5 --- /dev/null +++ b/docs/modules/scriptless-automation/pages/best-practices.adoc @@ -0,0 +1,103 @@ += Best practices for Scriptless automation + +== Overview + +Best practices help improve the reliability and consistency of Scriptless automation. + +These guidelines are not required to run Scriptless tests, but following them can reduce execution issues and minimize the need for remediation as test coverage expands. + +== Baseline session best practices + +Baseline sessions are most effective when they reflect a clear and stable test flow. + +When creating a baseline session: + +* Enable session capture by selecting the *Play* button before interacting with the application to ensure accurate element detection. +* Use a physical keyboard instead of the virtual device keyboard. +* Follow a single, intentional path through the application. +* Be precise and deliberate when interacting with the application. For example, tap the center of a button instead of the edge. +* Dismiss pop-ups by selecting the close control instead of tapping outside the pop-up. +* Avoid rapid swiping or tapping. Wait at least 2 seconds between actions. +* Avoid tapping auto-corrected text. +* Use supported interactions only. + +Baseline sessions that are focused and repeatable are easier to reuse and more reliable during execution. + +== Unsupported actions + +Scriptless automation does not support the following actions: + +* Double tap +* Double Home +* Pinch or zoom +* Screenshot +* Set time zone +* Image injection +* Custom gestures + +== Test design considerations + +Keeping tests concise improves maintainability. + +Shorter test flows are easier to understand, execute, and troubleshoot. Instead of extending a single test to cover many variations, related tests can be grouped and executed together. + +If a test requires frequent adjustments to continue running, recreating the baseline session may be more effective than repeatedly refining an unstable definition. + +== Execution consistency + +Consistency across devices and environments improves execution reliability. + +When possible: + +* Use similar device types, operating system versions, and screen layouts. +* Follow Kobiton device setup instructions before running Scriptless tests. +* Keep test devices in light mode. + +Greater variation across devices may increase the likelihood of execution differences that require review. + +Greater variation across devices may increase the likelihood of execution differences that require review. + +== Scaling test execution + +Tests can be executed across many devices once a baseline has proven reliable. + +Expanding execution incrementally allows issues to surface early and keeps results easier to interpret. Grouping related tests helps maintain clarity as execution scales. + +== Reducing remediation effort + +Remediation is an expected part of automated execution, but its frequency can often be reduced. + +Before starting a new test run, review the test case and update any steps that no longer match the current application flow. + +Clear baseline sessions, consistent navigation paths, and stable application states all contribute to smoother execution. Repeated remediation across many steps may indicate that the underlying test flow no longer reflects current application behavior. + +== Workflows to avoid + +Some workflows are not suited for Scriptless automation. + +Avoid using Scriptless automation for apps or flows with: + +* First-run states that differ from later app launches +* Dynamic content that changes frequently or unpredictably +* Random pop-ups, such as promotional messages +* Non-English app text +* Complex gesture-based interactions +* Gaming apps + +These workflows may require script-based automation for greater control. + +== Unsupported frameworks and environments + +Scriptless automation does not support: + +* Espresso +* UIAutomator +* XCUITest +* GameDriver +* Virtual devices + +== Related pages + +* xref:scriptless-automation:baseline-session.adoc[Create a baseline session] +* xref:scriptless-automation:run-scriptless.adoc[Run a Scriptless test] +* xref:test-management:remediations.adoc[Remediations] diff --git a/docs/modules/scriptless-automation/pages/create-a-baseline-session.adoc b/docs/modules/scriptless-automation/pages/create-a-baseline-session.adoc deleted file mode 100644 index 5ed99c100..000000000 --- a/docs/modules/scriptless-automation/pages/create-a-baseline-session.adoc +++ /dev/null @@ -1,50 +0,0 @@ -= Create a baseline session for Scriptless Automation -:navtitle: Create a baseline session - -Learn how to create a baseline session for Scriptless Automation. After you create a baseline session, you can use the xref:run-scriptless-with-the-kobiton-portal.adoc[Kobiton portal] or xref:run-scriptless-with-the-kobiton-api.adoc[Kobiton API] to rerun your test steps on the same device or on completely different devices--_without_ writing any test scripts. - -include::manual-testing:partial$start-a-manual-session.adoc[] - -[#_optional_install_an_app] -== Optional: install an app - -If you plan on testing an app, first you'll need to install it on your device. Select *Install Apps* to get stared. - -image:manual-testing:install-apps-context.png[width=1000,alt="Install Apps"] - -[IMPORTANT] - -==== -Make sure your app aligns with the Scriptless xref:scriptless-automation:scriptless-requirements.adoc#_application_requirements[application requirements] before proceeding. -==== - -You can upload a xref:apps:supported-filetypes.adoc[supported filetype] from your computer, install an app from a URL, or choose an app from xref:apps:manage-apps.adoc[your app repository]. - -image:manual-testing:install-apps-closeup.png[width=750, alt="A close-up of the different app installation options during a manual session."] - -== Enable synchronous inventory capture - -To ensure each action is fully captured for your baseline session, enable xref:manual-testing:device-controls.adoc#_synchronous_inventory_capture[synchronous inventory capture] by selecting the *Play* icon. - -image:manual-testing:capture-context.png[width=1000,alt="Enable Synchronous inventory capture by selecting the Play icon"] - -[#_use_supported_actions] -== Use supported actions - -include::scriptless-requirements.adoc[tag=supported-actions] - -== Follow scriptless best practices - -For best results, follow the xref:scriptless-automation:scriptless-best-practices.adoc[] while running your tests. - -== Exit your session - -When you're finished testing, select the *X* icon. - -image:select-x-to-end-session-context.png[width=1000,alt="Select X to end a session"] - -Select *Exit Session*. - -image:exit-session-context.png[width=500,alt="Exit Session pop up"] - -Now you can use this session to run Scriptless Automation using the xref:run-scriptless-with-the-kobiton-portal.adoc[Kobiton portal] or xref:run-scriptless-with-the-kobiton-api.adoc[Kobiton API]. diff --git a/docs/modules/scriptless-automation/pages/get-a-session-id/using-the-kobiton-api.adoc b/docs/modules/scriptless-automation/pages/get-a-session-id/using-the-kobiton-api.adoc deleted file mode 100644 index 3f09c1b34..000000000 --- a/docs/modules/scriptless-automation/pages/get-a-session-id/using-the-kobiton-api.adoc +++ /dev/null @@ -1 +0,0 @@ -include::automation-testing:get-a-session-id/using-the-kobiton-api.adoc[] \ No newline at end of file diff --git a/docs/modules/scriptless-automation/pages/get-a-session-id/using-the-kobiton-portal.adoc b/docs/modules/scriptless-automation/pages/get-a-session-id/using-the-kobiton-portal.adoc deleted file mode 100644 index fb853add4..000000000 --- a/docs/modules/scriptless-automation/pages/get-a-session-id/using-the-kobiton-portal.adoc +++ /dev/null @@ -1 +0,0 @@ -include::automation-testing:get-a-session-id/using-the-kobiton-portal.adoc[] diff --git a/docs/modules/scriptless-automation/pages/index.adoc b/docs/modules/scriptless-automation/pages/index.adoc index b7a14981e..fdcb2411e 100644 --- a/docs/modules/scriptless-automation/pages/index.adoc +++ b/docs/modules/scriptless-automation/pages/index.adoc @@ -1,10 +1,63 @@ -= Scriptless Automation -:navtitle: Scriptless Automation += Scriptless automation -Automate your tests without writing scripts. +== Overview -image:automation-testing:automation-testing-index-context.png[width=1000,alt="A context to Scriptless Automation"] +Scriptless automation enables automated test execution based on interactions captured during manual sessions. Tests are generated from real device usage and can be executed repeatedly without requiring users to write or maintain automation scripts. -== In this section +Within the Kobiton platform, this capability builds on manual testing by capturing user interactions in a structured form and reusing them for repeatable execution across devices. -include::../nav.adoc[lines=2..] +Scriptless automation is used to reduce the effort required to run repetitive tests while preserving visibility into how tests are executed on real devices. + +== How it works + +Scriptless automation is based on capturing test behavior once and reusing it for execution. + +=== Manual sessions as the source + +Testing begins with a manual session performed on a real device through the Kobiton platform. During the session, user interactions with the application are captured. + +=== Tests created from captured interactions + +Captured interactions are converted into test definitions that represent a complete test flow. These definitions are stored and managed independently of execution. + +=== Tests executed without user-written scripts + +When a test is run, the platform executes the captured interactions automatically. Automation scripts are generated and managed by the platform; users do not need to author or maintain code to run tests. + +== Requirements + +Scriptless automation requires the following: + +* An active Kobiton account with access to Test Management +* A supported mobile platform (Android or iOS) +* A supported application context, as described below + +=== Application under test + +The application under test must be accessible in a form supported for Scriptless automation. + +Depending on the application type: + +* *Native or hybrid mobile apps* must be uploaded to the Kobiton App Repository before creating or running tests. Direct app upload during test execution is not supported. +* *Web applications* can be tested through supported mobile browsers without uploading an app. +* *Preinstalled system applications* already available on the device (such as Clock or Calculator) can be tested without additional setup. + +== Limitations + +Scriptless automation tests are platform-specific. Separate tests are required for Android and iOS. + +Some application types and flows are not supported or are a poor fit for Scriptless automation, including: + +* Game engine–based applications, such as Unity-based apps +* Canvas-based applications, where UI elements are not represented as standard view hierarchies +* Highly dynamic interfaces that change structure frequently at runtime +* Authentication flows that rely on one-time credentials, such as OTP or multi-factor authentication + +In these scenarios, script-based automation or alternative testing approaches may be more appropriate. + +== Related pages + +* xref:scriptless-automation:baseline-session.adoc[Create a baseline session] +* xref:scriptless-automation:run-scriptless.adoc[Run a Scriptless test] +* xref:scriptless-automation:best-practices.adoc[Best practices] +* xref:test-management:index.adoc[Test management] diff --git a/docs/modules/scriptless-automation/pages/run-scriptless-api.adoc b/docs/modules/scriptless-automation/pages/run-scriptless-api.adoc new file mode 100644 index 000000000..d0ce79ca0 --- /dev/null +++ b/docs/modules/scriptless-automation/pages/run-scriptless-api.adoc @@ -0,0 +1,46 @@ += Run a Scriptless test using the API + +== Overview + +Test cases created from captured manual sessions can also be executed programmatically using the Kobiton API. + +API-based execution is typically used for advanced workflows, such as integrating test runs into automated pipelines or triggering tests as part of a larger system. The API provides an alternative to running tests through the platform UI. + +== When to use the API + +Using the API is appropriate when test execution needs to be: + +* Triggered automatically as part of a CI/CD pipeline +* Integrated with external systems or custom tooling +* Managed without manual interaction in the platform + +The API does not replace the platform UI. Test cases and test suites must still be created and managed through Test Management. + +== How API-based execution works + +API-based execution uses the same underlying test definitions as platform-based execution. + +Tests are created from baseline sessions and stored in Test Management. When triggered through the API, these existing test definitions are executed on the selected devices using the same execution logic as platform-initiated runs. + +Execution results are stored in Test Management and can be reviewed or remediated in the platform UI. + +== Requirements + +To run tests using the API, the following are required: + +* An active Kobiton account with API access +* Existing test cases or test suites +* Familiarity with API-based workflows + +== API documentation + +Detailed information about available endpoints, request formats, and authentication is provided in the API documentation. + +* To trigger and manage test runs: link:https://api.kobiton.com/v2/docs#tag/TestRunAPI[Test Run API] +* To manage test cases: link:https://api.kobiton.com/v2/docs#tag/TestCaseAPI[Test Case API] + +== Related pages + +* xref:scriptless-automation:baseline-session.adoc[Create a baseline session] +* xref:test-management:index.adoc[Test management] +* xref:test-management:remediations.adoc[Remediations] diff --git a/docs/modules/scriptless-automation/pages/run-scriptless-with-the-kobiton-api.adoc b/docs/modules/scriptless-automation/pages/run-scriptless-with-the-kobiton-api.adoc deleted file mode 100644 index 0f1f8d569..000000000 --- a/docs/modules/scriptless-automation/pages/run-scriptless-with-the-kobiton-api.adoc +++ /dev/null @@ -1,176 +0,0 @@ -= *Under construction:* Run scriptless with the Kobiton API -:navtitle: Run scriptless with the Kobiton API - -The content of this page is being revamped. Check back soon! - -Meanwhile, you can xref:run-scriptless-with-the-kobiton-portal.adoc[run scriptless with the Kobiton portal] instead. - -//// - -Learn how to run Scriptless Automation using the Kobiton API. For more information, see link:https://api.kobiton.com/docs/#start-scriptless-automation[Kobiton API v1]. - -[#_before_you_start] -== Before you start - -You'll need to complete the following: - -* xref:create-a-baseline-session.adoc[Create one or more baseline sessions]. -* Get the session ID for your baseline session(s) using the xref:get-a-session-id/using-the-kobiton-portal.adoc[Kobiton portal] or xref:get-a-session-id/using-the-kobiton-api.adoc[Kobiton API]. -* xref:profile:manage-your-api-credentials.adoc[Get your API credentials]. - -[#_configure_your_authorization_header] -== Configure your authorization header - -include::automation-testing:partial$configure-your-authorization-header.adoc[] - -[#_get_your_parameter_values] -== Get your parameter values - -Determine which link:https://api.kobiton.com/docs/#start-scriptless-automation[Scriptless Automation parameters] you want to use during your session, then get the necessary values. The `/revisitPlans/start` endpoint supports up to 11 parameters, but only the following are required: - -[cols="1,1,1,3"] -|=== -|Parameter |Type |Required? |Description - -|`exploringSessionIds` -|`[integer]` -|Always -|A list of one or more xref:get-a-session-id/using-the-kobiton-api.adoc[baseline session IDs], where each baseline session is run as its own Scriptless Automation session. Up to 10 baseline session IDs can be assigned; however, the same device platform must be used for each session (such as `iOS` or `Android`). - -|`deviceSelections` -|`[object]` -|Yes, if `deviceBundleId` is unassigned -|An array of one or more devices, where each device is assigned a `deviceCapabilities` object (required) and `dataSetId` (optional). For more information, see link:https://api.kobiton.com/docs/#start-scriptless-automation[Scriptless Automation parameters]. - -|`deviceBundleId` -|`[integer]` -|Yes, if `deviceSelections` is unassigned -|A list of one or more device bundle IDs where each device will run a scriptless session for each session in `exploringSessionIds`. Up to six device bundle IDs can be assigned. -|=== - -== Make a `POST` request - -In the terminal, make a `POST` request using your xref:_configure_your_authorization_header[authorization header] and xref:_get_your_parameter_values[parameter values]. - -[source,shell] ----- -curl -X POST https://api.kobiton.com/v1/revisitPlans/start \ - -H 'Authorization: Basic ${yourBase64String}' \ - -H 'Content-Type: application/json' \ - -H 'Accept: application/json' \ - -d '{ - "exploringSessionIds": [${sessionId}], - "deviceSelections": [ - { - "dataSetId": ${dataSetId}, - "deviceCapabilities": [ - { - "deviceName": "${deviceName}", - "platformVersion": "${platformVersion}", - "deviceSource": "${deviceSource}" - } - ], - } - ], - "appPath": "${appPath}", - "deviceBundleId": [${deviceBundleId}], - "runAllDevicesInBundle": ${runAllDevicesInBundleBoolean}, - "testCaseIds": [${testCaseId}] - }' ----- - -A successful `POST` response will return a `200` status, along with the following information: - -[source,shell] ----- -[ - { - "exploringSessionId": ${exploringSessionId}, - "deviceCapabilities": [ - { - "dataSetId": ${dataSetId}, - "deviceName": "${deviceName}", - "platformVersion": "${platformVersion}", - "deviceSource": "${deviceSource}" - } - ], - "executionLink": "https://portal.kobiton.com/${pathToYourSession}" - } -] ----- - -When you're finished, use the URL assigned to `testRunDetailLink` to review your session in xref:session-explorer:index.adoc[Session Explorer]. - -== Example `POST` request - -Here's an example `POST` request where multiple devices will each download an app and test the app three different times using three different baseline sessions. - -.Example -[source,shell] ----- -curl -X POST https://api.kobiton.com/v1/revisitPlans/start \ - -H 'Authorization: Basic dGVzdHVzZXI6MTIzZWQtMTIzZmFjLTkxMzdkY2E=' \ - -H 'Content-Type: application/json' \ - -H 'Accept: application/json' \ - -d '{ - "exploringSessionIds": [1, 2, 3], - "deviceSelections": [ - { - "dataSetId": 99, - "deviceCapabilities": [ - { - "deviceName": "Galaxy S8", - "platformVersion": "*", - "deviceSource": "KOBITON" - }, - { - "deviceName": "*S9*", - "platformVersion": "10.0.0", - "deviceSource": "KOBITON" - } - ] - }, - { - "dataSetId": 100, - "deviceCapabilities": { - "deviceName": "Nokia*", - "platformVersion": "11.0.0", - "deviceSource": "KOBITON" - } - } - ], - "appPath": "kobiton-store:v100", - "deviceBundleId": [1, 2, 3 ], - "runAllDevicesInBundle": true, - "testCaseIds": [1, 2, 3] - }' ----- - -If the `POST` request is successful, the following will be returned: - -.Example -[source,shell] ----- -[ - { - "exploringSessionId": 0, - "deviceCapabilities": [ - { - "dataSetId": 99, - "deviceName": "Galaxy S9", - "platformVersion": "10.0.0", - "deviceSource": "KOBITON" - }, - { - "dataSetId": 100, - "deviceName": "Nokia 5.3", - "platformVersion": "11.0.0", - "deviceSource": "KOBITON" - } - ], - "executionLink": "https://portal.kobiton.com/sessions/100/plan/executions" - } -] ----- - -//// diff --git a/docs/modules/scriptless-automation/pages/run-scriptless-with-the-kobiton-portal.adoc b/docs/modules/scriptless-automation/pages/run-scriptless-with-the-kobiton-portal.adoc deleted file mode 100644 index 03295ade3..000000000 --- a/docs/modules/scriptless-automation/pages/run-scriptless-with-the-kobiton-portal.adoc +++ /dev/null @@ -1,82 +0,0 @@ -= Run scriptless with the Kobiton portal -:navtitle: Run scriptless with the Kobiton portal - -Learn how to run Scriptless Automation using the Kobiton portal. - -== Before you start - -You'll need to complete the following: - -* xref:create-a-baseline-session.adoc[Create a baseline session]. -* xref:test-management:create-a-test-case.adoc[] using your baseline session. - -== Open Scriptless Automation - -In *Sessions*, search for your baseline session, then select the session. - -image:session-explorer:search-session-id-closeup.png[width=1000,alt="Session search screen"] - -[NOTE] -For more information, see xref:session-explorer:session-metadata.adoc[]. - -Select *Automated Test Case*. - -image:scriptless-automation:scriptless-automated-test-case-button.png[width=1000,alt="The session overview page with the Automated Test Case button"] - -Select *Rerun*. - -image:scriptless-automation:scriptless-rerun-button.png[width=1000,alt="Select Rerun in the Automated Test Case screen"] - -[TIP] -For best results, select Rerun when the message _Congratulations! Your test case is good to go now_ appears on screen. If a different message is showing, you may need to xref:test-management:remediation/annotate-a-test-step.adoc[annotate the test steps]. - -== Configure your scriptless session - -=== Device Bundles - -In the *Rerun Configurations* window, select *Device Bundles*. - -image:scriptless-automation:scriptless-rerun-configurations-device-bundles-context.png[width=600,alt="The select device bundles dropdown list"] - -Select the dropdown and choose a bundle. - -image:scriptless-automation:scriptless-select-device-bundles.png[width=600,alt="The select device bundles dropdown list"] - -If you'd like to remove any devices for this session, de-select the checkbox next to each device. - -image:scriptless-automation:scriptless-deselect-device-from-bundles.png[width=1000,alt="A device is de-selected from the device bundles"] - -[NOTE] -If you'd like to make more changes to this bundle, select *Configure Bundle*. For more information, see xref:organization:device-bundles/manage-device-bundles.adoc[]. - -=== Data driven testing - -Data-driven testing allows you to define custom data sets and replace values from your baseline session. In the *Rerun Configurations* window, select *Data Driven Testing*. - -image:scriptless-automation:rerun-configurations-data-driven-testing-context.png[width=1000,alt="The Data Driven Testing section under Rerun Configurations"] - -Select the *+* button to add new test data. - -image:scriptless-automation:data-driven-new-dataset.png[width=1000,alt="Select the plus icon to add new test data"] - -The new test data is only applied to a specific device in the bundle. Choose the device to apply by selecting it from the *Assign Device* dropdown. - -image:scriptless-automation:data-driven-assign-device.png[width=500,alt="The Assign Device dropdown"] - -To replace a value used for a specific test step, select the test step with a value input action, then enter a new value. - -image:scriptless-automation:data-driven-replace-value.png[width=500,alt="Replacing a value in a test step"] - -Once a value has been modified, it will be marked as *Edited*. - -image:scriptless-automation:data-driven-test-step-edited.png[width=500,alt="The value is marked as edited"] - -== Run Scriptless Automation - -When you're finished configuring your scriptless session, select *Rerun*. - -image:scriptless-automation:rerun-configurations-data-driven-testing-context.png[width=1000,alt="Select Rerun after finishing the configurations"] - -You'll be redirected to your test run where you can view live updates for your run. - -image:scriptless-automation:test-run-view.png[width=1000,alt="The test run view"] diff --git a/docs/modules/scriptless-automation/pages/run-scriptless.adoc b/docs/modules/scriptless-automation/pages/run-scriptless.adoc new file mode 100644 index 000000000..6ee379b16 --- /dev/null +++ b/docs/modules/scriptless-automation/pages/run-scriptless.adoc @@ -0,0 +1,62 @@ += Run a Scriptless test + +== Overview + +Running a Scriptless test applies an existing test definition to one or more devices. Tests are executed automatically based on interactions previously captured during a manual session and converted to a test case. + +In the Kobiton platform, Scriptless tests are run through Test Management. Execution results are captured for review and can be rerun as needed. + +== Before you begin + +Before running a test, ensure the following: + +* A test case has been created from a baseline session +* The application under test is available in a supported application context (native app, web application, or preinstalled app) +* At least one compatible device is available + +== Run a test in the platform + +Tests are run from Test Management in the Kobiton platform. + +From Test Management, users can create a test run using a single test case or a test suite. During setup, the test definition, application version, and target devices are selected. + +Once a test run starts, the platform executes the captured interactions automatically on the selected devices. Test execution does not require user interaction while the run is in progress. + +== What happens during execution + +During execution, the platform performs the test case steps on each device. + +For each step, the system attempts to match the captured interaction to the current application state. Execution continues until all steps are completed or until user action is required. + +Execution progress and status are visible while the test is running. + +== Review test results + +After execution completes, results are available in xref:test-management:test-runs.adoc[Test Runs] under Test Management. + +Each test run includes: + +* Overall execution status +* Device-level execution details +* Validation results, such as Crash, Accessibility, and Response Time findings + +If execution cannot proceed for a particular step, the test run is marked as requiring action and must be remediated before execution can continue. + +== Rerun a test + +Test runs can be rerun using the same configuration. + +Reruns repeat the original execution using the same test definition and device selection. This is commonly used after issues are addressed or when validating execution behavior. + +== Advanced usage + +Tests can also be triggered programmatically using the Kobiton API. This approach is typically used for automated pipelines or advanced workflows. + +For details, see xref:scriptless-automation:run-scriptless-api.adoc[Run a Scriptless test using the API]. + +== Related pages + +* xref:scriptless-automation:baseline-session.adoc[Create a baseline session] +* xref:scriptless-automation:run-scriptless-api.adoc[Run a Scriptless test using the API] +* xref:test-management:index.adoc[Test management] +* xref:test-management:remediations.adoc[Remediations] diff --git a/docs/modules/scriptless-automation/pages/scriptless-best-practices.adoc b/docs/modules/scriptless-automation/pages/scriptless-best-practices.adoc deleted file mode 100644 index d936e0424..000000000 --- a/docs/modules/scriptless-automation/pages/scriptless-best-practices.adoc +++ /dev/null @@ -1,28 +0,0 @@ -= Scriptless best practices -:navtitle: Scriptless best practices - -== Order of app installation - -If apps need to be installed or uploaded in the session, make sure to install/upload the apps first before performing any other actions. - -== Performance - -The following actions may impact Scriptless Automation performance - -* Clicking very quickly. For best results, please wait three seconds between two manual actions. This wait will ensure the screen has loaded completely when you make another “click” action. -* Swiping on map -* Clicking outside the pop-up to close it. Instead - use the Back button in the manual toolbar to close the pop-up -* Tapping on text prediction above the virtual keyboard -* Typing text into number input only -* Typing string that has length more than limit when editing text - -== General Limitations - -* Avoid interacting too fast during manual execution as every action performed during manual execution will take a minor time for inventory capturing, especially when testing on Mobile Browser or Hybrid apps. - -[TIP] -To ensure every action could be properly captured data, enable xref:manual-testing:device-controls.adoc#_synchronous_inventory_capture[synchronous inventory capture] during manual execution. - -* Avoid tapping on a blank space, e.g. to close a pop-up, to un-focus on a text box, etc. This tap could cause confusion for AI when finding elements and impact the accuracy of revisit execution. -* Avoid tapping on the device’s virtual keyboard. The process of capturing elements within the virtual keyboard still needs further improvements. -* Avoid going back to the Home screen when editing a text field. \ No newline at end of file diff --git a/docs/modules/scriptless-automation/pages/scriptless-requirements.adoc b/docs/modules/scriptless-automation/pages/scriptless-requirements.adoc deleted file mode 100644 index 2c7e2cd59..000000000 --- a/docs/modules/scriptless-automation/pages/scriptless-requirements.adoc +++ /dev/null @@ -1,126 +0,0 @@ -= Scriptless requirements -:navtitle: Scriptless requirements - -Learn about the requirements to convert a manual or automation session into Automated Test Case for Scriptless. - -== Supported actions - -// tag::supported-actions[] - -Only certain actions are supported for scriptless baseline sessions. Refer to the following table while running your tests: - -[cols="1,1,1,1"] -|=== -|Action |Supported? |Android |iOS - -|Touch -|✔ -|✔ -|✔ - -|Swipe -|✔ -|✔ -|✔ - -|Device soft keys -|✔ -| -|✔ - -|Virtual keyboard (ASCII characters) -|✔ -|✔ -b|✔ * - -_Only English is supported, excluding the emoji keyboard._ - -|Physical keyboard -|✔ -|✔ -|✔ - -|Home (toolbar) -|✔ -|✔ -|✔ - -|Power (toolbar) -|✔ -|✔ -|✔ - -|Recent app (toolbar) -|✔ -| -|✔ - -|Back (toolbar) -|✔ -| -|✔ - -|Long press -|✔ -|✔ -|✔ - -|Double tap -| -| -| - -|Double press home -| -| -| - -|Pin to zoom -| -| -| - -|Take screenshot -| -| -| - -|Set timezone -| -| -| - -|Copy & paste (copy to clipboard) -| -| -| - -|Input sensitive data (password, passcode) -| -| -| -|=== - -If an unsupported action is performed, the Scriptless Automation stops capturing, and subsequent steps after the unsupported action are not captured and executable in rerun/revisit or in the auto-generated scripts. - -image:scriptless-automation:scriptless-unsupported-action.png[width=1000,alt="An unsupported action performed in a manual session"] - -In the Session Explorer, the unsupported actions are highlighted in red: - -image:scriptless-automation:unsupported-action-session-explorer.png[width=1000,alt="An unsupported action displayed in Session Explorer"] - -In the Automated Test Case, only test steps before the unsupported action are included: - -image:scriptless-automation:unsupported-action-automated-test-case.png[width=1000,alt="An unsupported action displayed in Automated Test Case"] - -// end::supported-actions[] - -== Application requirements - -The following requirements pertain to the application(s) installed and/or opened in the manual or automation session: - -* Must not contain dynamic content. Examples: news apps, games, etc. - -* Must not contain CAPTCHA verification flow. - -* Must enable `.setWebContentDebuggingEnable` for Android apps. If this configuration is disabled, Kobiton cannot access the HTML source of the application. Visit https://developer.chrome.com/docs/devtools/remote-debugging/webviews/#configure_webviews_for_debugging[this link] for more information. \ No newline at end of file diff --git a/docs/modules/test-management/images/blocker-remediation-modal.png b/docs/modules/test-management/images/blocker-remediation-modal.png new file mode 100644 index 000000000..1d4a2ce2c Binary files /dev/null and b/docs/modules/test-management/images/blocker-remediation-modal.png differ diff --git a/docs/modules/test-management/images/camera-roll-resolve.png b/docs/modules/test-management/images/camera-roll-resolve.png new file mode 100644 index 000000000..b67aaabb2 Binary files /dev/null and b/docs/modules/test-management/images/camera-roll-resolve.png differ diff --git a/docs/modules/test-management/images/camera-roll-view.png b/docs/modules/test-management/images/camera-roll-view.png new file mode 100644 index 000000000..58bdcb62f Binary files /dev/null and b/docs/modules/test-management/images/camera-roll-view.png differ diff --git a/docs/modules/test-management/images/camera-roll.png b/docs/modules/test-management/images/camera-roll.png new file mode 100644 index 000000000..cbf2d9c58 Binary files /dev/null and b/docs/modules/test-management/images/camera-roll.png differ diff --git a/docs/modules/test-management/images/genappium-language.png b/docs/modules/test-management/images/genappium-language.png new file mode 100644 index 000000000..fb8be309d Binary files /dev/null and b/docs/modules/test-management/images/genappium-language.png differ diff --git a/docs/modules/test-management/images/genappium-location.png b/docs/modules/test-management/images/genappium-location.png new file mode 100644 index 000000000..2ddf180fb Binary files /dev/null and b/docs/modules/test-management/images/genappium-location.png differ diff --git a/docs/modules/test-management/images/live-remediation-countdown.png b/docs/modules/test-management/images/live-remediation-countdown.png new file mode 100644 index 000000000..a5c88d344 Binary files /dev/null and b/docs/modules/test-management/images/live-remediation-countdown.png differ diff --git a/docs/modules/test-management/images/popup-handler.png b/docs/modules/test-management/images/popup-handler.png new file mode 100644 index 000000000..ee7d2727f Binary files /dev/null and b/docs/modules/test-management/images/popup-handler.png differ diff --git a/docs/modules/test-management/images/rem-test-step.png b/docs/modules/test-management/images/rem-test-step.png new file mode 100644 index 000000000..91c3b6265 Binary files /dev/null and b/docs/modules/test-management/images/rem-test-step.png differ diff --git a/docs/modules/test-management/images/se-window.png b/docs/modules/test-management/images/se-window.png new file mode 100644 index 000000000..772449eff Binary files /dev/null and b/docs/modules/test-management/images/se-window.png differ diff --git a/docs/modules/test-management/images/test-run-config.png b/docs/modules/test-management/images/test-run-config.png new file mode 100644 index 000000000..fb0396c9c Binary files /dev/null and b/docs/modules/test-management/images/test-run-config.png differ diff --git a/docs/modules/test-management/images/test-run-list-actions.png b/docs/modules/test-management/images/test-run-list-actions.png new file mode 100644 index 000000000..b7b29e858 Binary files /dev/null and b/docs/modules/test-management/images/test-run-list-actions.png differ diff --git a/docs/modules/test-management/images/test-run-result-needs-action.png b/docs/modules/test-management/images/test-run-result-needs-action.png new file mode 100644 index 000000000..89afd61ca Binary files /dev/null and b/docs/modules/test-management/images/test-run-result-needs-action.png differ diff --git a/docs/modules/test-management/images/test-run-results-summary.png b/docs/modules/test-management/images/test-run-results-summary.png new file mode 100644 index 000000000..7cfe5bf02 Binary files /dev/null and b/docs/modules/test-management/images/test-run-results-summary.png differ diff --git a/docs/modules/test-management/images/test-run-setup.png b/docs/modules/test-management/images/test-run-setup.png new file mode 100644 index 000000000..7eb462f43 Binary files /dev/null and b/docs/modules/test-management/images/test-run-setup.png differ diff --git a/docs/modules/test-management/images/test-suite-list.png b/docs/modules/test-management/images/test-suite-list.png new file mode 100644 index 000000000..9aca6c4e2 Binary files /dev/null and b/docs/modules/test-management/images/test-suite-list.png differ diff --git a/docs/modules/test-management/images/tm-edit-text.png b/docs/modules/test-management/images/tm-edit-text.png new file mode 100644 index 000000000..276e5aa95 Binary files /dev/null and b/docs/modules/test-management/images/tm-edit-text.png differ diff --git a/docs/modules/test-management/images/tm-newtcscreen.png b/docs/modules/test-management/images/tm-newtcscreen.png new file mode 100644 index 000000000..47150de25 Binary files /dev/null and b/docs/modules/test-management/images/tm-newtcscreen.png differ diff --git a/docs/modules/test-management/images/tm-remediate.png b/docs/modules/test-management/images/tm-remediate.png new file mode 100644 index 000000000..c3c501964 Binary files /dev/null and b/docs/modules/test-management/images/tm-remediate.png differ diff --git a/docs/modules/test-management/images/view-session-detail.png b/docs/modules/test-management/images/view-session-detail.png new file mode 100644 index 000000000..fe02aa95a Binary files /dev/null and b/docs/modules/test-management/images/view-session-detail.png differ diff --git a/docs/modules/test-management/nav.adoc b/docs/modules/test-management/nav.adoc index 592843753..f63036424 100644 --- a/docs/modules/test-management/nav.adoc +++ b/docs/modules/test-management/nav.adoc @@ -1,9 +1,8 @@ .xref:index.adoc[] -* xref:test-management:create-a-test-case.adoc[] -* xref:test-management:manage-test-steps.adoc[] -* xref:test-management:remediation/index.adoc[] -** xref:test-management:remediation/annotate-a-test-step.adoc[] -** xref:test-management:remediation/crash-remediation.adoc[] -** xref:test-management:remediation/element-selection-remediation.adoc[] -** xref:test-management:remediation/ignore-remediation.adoc[] +* xref:test-management:test-cases.adoc[] +* xref:test-management:generate-appium.adoc[] +* xref:test-management:test-runs.adoc[] +* xref:test-management:test-suites.adoc[] +* xref:test-management:test-reruns.adoc[] +* xref:test-management:remediations.adoc[] diff --git a/docs/modules/test-management/pages/create-a-test-case.adoc b/docs/modules/test-management/pages/create-a-test-case.adoc deleted file mode 100644 index 8d3c434b8..000000000 --- a/docs/modules/test-management/pages/create-a-test-case.adoc +++ /dev/null @@ -1,31 +0,0 @@ -= Create a test case -:navtitle: Create a test case - -Learn how to create a test case so you can run xref:scriptless-automation:index.adoc[Scriptless Automation]. - -== Before you start - -You'll need to complete the following: - -* xref:scriptless-automation:create-a-baseline-session.adoc[Create a baseline session]. -* If applicable, xref:remediation/annotate-a-test-step.adoc[annotate any flagged test steps]. - -== Create a test case - -In *Sessions*, xref:session-explorer:search-for-a-session.adoc[search for a session], then select a session. - -image:search-for-a-session-context.png[width=1000,alt="Select Sessions and search for a session"] - -Select *Automated Test Case*. - -image:select-automated-test-case-context.png[width=1000,alt="Select Automated Test Case"] - -If your test case has any issues, you'll need to xref:remediation/annotate-a-test-step.adoc[annotate your flagged test steps] first. When your test case is good to go, you'll see the following message: - -image:test-case-ready-to-go-context.png[width=1000,alt="Test case after all annotating"] - -When you're ready, select *Convert to Test Case*. - -image:convert-to-test-case-context.png[width=1000,alt="Click Convert to Test case"] - -If you'd like to make changes to your test case before running Scriptless Automation, see xref:manage-test-steps.adoc[]. diff --git a/docs/modules/test-management/pages/generate-appium.adoc b/docs/modules/test-management/pages/generate-appium.adoc new file mode 100644 index 000000000..346ceaf74 --- /dev/null +++ b/docs/modules/test-management/pages/generate-appium.adoc @@ -0,0 +1,56 @@ += Generate an Appium script + +== Overview + +An Appium script can be generated from an existing Test Case created from a baseline session. + +The generated script reflects the interactions captured during the manual session and can be used in external, script-based automation workflows. + +Generating an Appium script is optional. Running Tests in the platform does not require exporting scripts. + +== Before you begin + +Before generating an Appium script: + +* A baseline session must be completed +* The baseline session must be converted into a Test Case +* Any required Test Case remediation must be resolved + +== Export an Appium script + +Appium scripts are generated from the Test Case view in the platform. + +From the Test Case: + +* Open the Test Case associated with the baseline session +* Select the option to export an Appium script +* Choose a supported language and Test framework +* Download the generated script + +The exported script includes the actions and element interactions captured during the baseline session. + +image::genappium-location.png[width=700,alt="Generate Appoum script from Test Management screen"] + +== Supported languages and frameworks + +Appium scripts can be generated using the following language and framework combinations: + +* Java (TestNG) +* Java (JUnit) +* Node.js (Mocha) +* C# (NUnit) +* Python (pyTest) + +image::genappium-language.png[width=700,alt="Generate Appoum script to language and framework"] + +== Using the generated script + +The generated Appium script can be extended, modified, or integrated into existing automation frameworks. + +Changes made to the script outside the platform do not affect the original Test Case or future Test runs executed in the platform. + +== Related pages + +* xref:test-management:test-cases.adoc[Test Cases] +* xref:scriptless-automation:baseline-session.adoc[Create a baseline session] +* xref:test-management:index.adoc[Test Management] diff --git a/docs/modules/test-management/pages/index.adoc b/docs/modules/test-management/pages/index.adoc index e3bc6a958..0398ad578 100644 --- a/docs/modules/test-management/pages/index.adoc +++ b/docs/modules/test-management/pages/index.adoc @@ -1,15 +1,52 @@ -= Test management -:navtitle: Test management -:page-aliases: test-management:validation:index.adoc, test-management:validation:color-text-validation.adoc, test-management:validation:performance-validation.adoc, test-management:validation:text-validation.adoc += Test Management -NOTE: Text color validation, performance validation, and text validation are no longer supported. +== Overview -Manage test cases for you or your team. +Test Management provides the structure used to organize, execute, and review automated tests created from manual sessions. -image:test-management:test-case-management-context.png[width=1000, alt="The test management context"] +Within the Kobiton platform, Test Management is where test cases are stored, grouped, and reused. It also provides visibility into test execution and results across devices and application versions. -== In this section +== How Test Management fits into testing -include::../nav.adoc[lines=2..] +Test Management separates test definition from test execution. +Test cases represent captured test flows derived from baseline sessions. These test cases can be executed repeatedly without recreating the original session. +Each execution produces a test run, which records the outcome of running a test case or test suite on one or more devices. Test runs and their results are tracked independently of the test definitions they use. + +== Core objects + +Test Management is built around a small set of core objects. + +=== Test cases + +Reusable definitions created from baseline sessions. Test cases describe what should be tested. + +=== Test runs + +Individual executions of a test case or test suite. Test runs capture results and execution details. + +=== Test suites + +Collections of related test cases grouped for execution and organization. + +=== Test reruns + +Repeat executions of a previous test run using the same configuration, typically used for validation. + +== What’s next + +From Test Management, users can: + +* Create and manage test cases +* Run tests across devices +* Review and remediate execution results +* Rerun tests after remediation +* Generate Appium scripts for automation testing + +== Related pages + +* xref:test-management:test-cases.adoc[Test cases] +* xref:test-management:test-runs.adoc[Test runs] +* xref:test-management:test-suites.adoc[Test suites] +* xref:test-management:test-reruns.adoc[Test reruns] diff --git a/docs/modules/test-management/pages/manage-test-steps.adoc b/docs/modules/test-management/pages/manage-test-steps.adoc deleted file mode 100644 index 3b3b6f72b..000000000 --- a/docs/modules/test-management/pages/manage-test-steps.adoc +++ /dev/null @@ -1,34 +0,0 @@ -= Manage test steps -:navtitle: Manage test steps - -Learn how to manage the test steps, so you can improve your test case before running Scriptless Automation. - -== Before you start - -You'll need to xref:test-management:create-a-test-case.adoc[create a test case] from a baseline session. - -== Open a test case - -In *Sessions*, xref:session-explorer:search-for-a-session.adoc[search for a session], then select a session. - -image:select-a-session-context.png[width=1000,alt="Search and select a session"] - -Select *Automation Test Case*. - -image:select-automated-test-case-context.png[width=1000,alt="Select Automated Test Case"] - -== Manage the test step - -=== Delete the test step - -You can delete one or more test steps at a time. Select the checkbox next to each test step you'd like to delete. - -image:delete-test-steps-context.png[width=1000,alt="Delete test steps"] - -When you're ready, select *Delete*. - -[IMPORTANT] -Deleting a test step cannot be undone. - -image:delete-test-step-popup-context.png[width=1000,alt=""] - diff --git a/docs/modules/test-management/pages/remediation/annotate-a-test-step.adoc b/docs/modules/test-management/pages/remediation/annotate-a-test-step.adoc deleted file mode 100644 index c76d6c5ae..000000000 --- a/docs/modules/test-management/pages/remediation/annotate-a-test-step.adoc +++ /dev/null @@ -1,37 +0,0 @@ -= Annotate a test step for Scriptless Automation -:navtitle: Annotate a test step - -Learn how to annotate a test step from your xref:scriptless-automation:create-a-baseline-session.adoc[baseline session], so you can... - -== Before you start - -You'll need to xref:scriptless-automation:create-a-baseline-session.adoc[create a baseline session]. - -== Annotate a test step - -In *Sessions*, xref:session-explorer:search-for-a-session.adoc[search for a session], then select the session. - -image:select-a-session-context.png[width=1000,alt="Search and select a session"] - -Select *Automation Test Case*. - -image:select-automated-test-case-context.png[width=1000,alt="Select Automated Test Case"] - -If one of your test steps were flagged, you'll see a warning similar to the following: - -image:annotate-warning-context.png[width=1000,alt="Test steps are flagged with the below warning"] - -Scroll through your test steps and look for any steps highlighted in red with the option to annotate. - -image:select-annotate-closeup.png[width=900,alt="Select Annotate"] - -Select *Annotate*. - -image:test-case-ready-to-go-context.png[width=1000,alt="Test cases are ready to go"] - -== Annotation options - -These are the different options for annotating a test step: - -** xref:test-management:remediation/crash-remediation.adoc[] -** xref:test-management:remediation/element-selection-remediation.adoc[] diff --git a/docs/modules/test-management/pages/remediation/crash-remediation.adoc b/docs/modules/test-management/pages/remediation/crash-remediation.adoc deleted file mode 100644 index e3a1efac0..000000000 --- a/docs/modules/test-management/pages/remediation/crash-remediation.adoc +++ /dev/null @@ -1,6 +0,0 @@ -= Crash remediation -:navtitle: Crash remediation - -If there's a crash during any execution, Scriptless Automation automatically detects it and provides the relevant crash log within the remediation process. Scriptless Automation offers remediation actions, like *Ignore* or *Report Bug*. - -For example, if a crash occurs on a Samsung S9 during a revisit, Scriptless Automation promptly captures it, initiating a remediation process. This process includes the specific crash log and vital device information, such as the OS, OS version, and resolution. This data equips testers with valuable insights to investigate the issue. diff --git a/docs/modules/test-management/pages/remediation/element-selection-remediation.adoc b/docs/modules/test-management/pages/remediation/element-selection-remediation.adoc deleted file mode 100644 index 54328d979..000000000 --- a/docs/modules/test-management/pages/remediation/element-selection-remediation.adoc +++ /dev/null @@ -1,12 +0,0 @@ -= Element selection remediation -:navtitle: Element selection remediation - -The User Interface (UI) of an element often changes across multiple devices. This can interrupt revisits on different devices because it's challenging to manage the appearance of an element through various device ratios, like 18:9, 17:9, and 16:9. But, with Scriptless Automation, you can easily handle these challenges through the UI without any scripting. - -image:scriptless-automation:ui-remediation-window.png[width=700,alt="UI Remediation window"] - -When revisiting on a Galaxy J5, a blocker is raised because a UI change covers and makes the **Login** button un-clickable. You can resolve this by manually selecting the **Login** button in the Scriptless Automation Remediation. This selection becomes the new 'baseline' for the test case, ensuring smooth execution. - -image:scriptless-automation:ui-remediation2.png[width=700,alt="Another view of UI Remediation window"] - -In situations where different screen ratios crop a list view cell, making it undetectable, this would typically result in a failed test case with other “Record & Playback” technologies. However, with Scriptless Automation, you can select the correct list view cell and submit the remediation to continue the test. diff --git a/docs/modules/test-management/pages/remediation/ignore-remediation.adoc b/docs/modules/test-management/pages/remediation/ignore-remediation.adoc deleted file mode 100644 index 2dc76053d..000000000 --- a/docs/modules/test-management/pages/remediation/ignore-remediation.adoc +++ /dev/null @@ -1,14 +0,0 @@ -= Ignore remediation -:navtitle: Ignore remediation - -If you come across a non-annotated remediation and are certain it's an application issue that needs fixing for the next release, you can use the *Ignore* option. This will exclude the execution from the Scriptless Automation process, so you won't face the remediation again. - -image:test-management:remediation-ignore-action.png[width=500,alt="The Ignore option under Blocker remediation's available actions"] - -[width="100%",cols="15%,85%",options="header"] -|=== -| Values | Description -| Ignore once | When this option is chosen, after submission, an execution will be re-run. The remediation will still appear if the issue persists. -| For 15 days | The execution will be excluded from the Scriptless Automation process for 15 days. -| Forever | The execution will be excluded from the Scriptless Automation process permanently. -|=== diff --git a/docs/modules/test-management/pages/remediation/index.adoc b/docs/modules/test-management/pages/remediation/index.adoc deleted file mode 100644 index 294d55ebd..000000000 --- a/docs/modules/test-management/pages/remediation/index.adoc +++ /dev/null @@ -1,11 +0,0 @@ -= Remediation -:navtitle: Remediation - -Remediate your test cases before running Scriptless Automation. - -== In this section - -* xref:test-management:remediation/annotate-a-test-step.adoc[] -* xref:test-management:remediation/crash-remediation.adoc[] -* xref:test-management:remediation/element-selection-remediation.adoc[] -* xref:test-management:remediation/ignore-remediation.adoc[] diff --git a/docs/modules/test-management/pages/remediations.adoc b/docs/modules/test-management/pages/remediations.adoc new file mode 100644 index 000000000..531d114c4 --- /dev/null +++ b/docs/modules/test-management/pages/remediations.adoc @@ -0,0 +1,144 @@ += Remediations + +== Overview + +Remediation addresses situations where automated execution cannot proceed as defined in the test case. + +During execution, the platform attempts to match captured interactions to the current application state on the device. When a step cannot be completed with sufficient confidence, execution pauses and user input is required before the test can continue. + +Remediation allows the test definition to be corrected without recreating the original baseline session. + +== When remediation is required + +When a Test Run encounters a blocker, its status indicates that user action is required. + +Remediation is triggered when an automated execution cannot reliably complete a step. This typically occurs when a UI element cannot be identified due to layout changes, screen size differences, or other variations across devices. + +When this happens, execution is paused until the blocker is resolved. + +Remediation is an expected part of maintaining reliable automated execution, particularly when tests are run across multiple devices or application versions. + +== Remediate a Test Run + +When a Test Run encounters a blocker, its status reflects that action is required. + +From the Test Run results, users can review where execution stopped and access the associated session. Remediation updates the test case definition so that future executions can proceed past the blocked step. + +After remediation is submitted, the test can be rerun using the same configuration. + +== Remediate in Session Explorer + +Session Explorer provides a detailed view of execution behavior. Use it to review where a Test Run stopped, compare the baseline to what the device showed during execution, and submit a correction to the Test Case. + +=== Remediate a blocked step + +From the Test Run Results page, select the *Action Needed* status on the affected run to open Session Explorer. In the Session Timeline located at the bottom of the screen, the blocked step is outlined in red and labeled *BLOCKED*. Select the blue checkmark icon for the blocked step to open the Blocker Remediation window. The window opens to the *Element Selection* tab by default. + +image::camera-roll-resolve.png[width=600,alt="Session Timeline with a red blocker icon and Resolve Blocker option"] + +You can also start remediation from an individual touch step in a Test Case. For more information, see xref:test-management:test-cases.adoc#_remediate_test_steps[Remediate test steps]. + +image::test-run-result-needs-action.png[width=600,alt="Test Run Results row with Actions Needed status"] + +=== Review the blocked step + +The Blocker Remediation modal displays the baseline screenshot alongside the screenshot from the revisit. The element captured during the baseline is highlighted in the *Baseline* panel. Hovering over the *Revisit* panel highlights the element closest to the cursor. + +Navigation buttons at the top of the modal move between test steps. Use these when a blocker appears on one step but the underlying issue started earlier in the flow. Only steps that use a touch action are reachable through navigation. + +image::blocker-remediation-modal.png[width=600,alt="Blocker Remediation modal with Baseline and Revisit panels and navigation buttons"] + +=== Submit a remediation + +. Open Blocker Remediation from the flagged step, or from the *⋮* menu on the relevant step. +. If the issue originated on an earlier step, use the navigation buttons to move to that step. +. Select the correct element from the *Revisit* screenshot. +. Select *Submit*. + +After submission, a new version of the Test Case is created. The updated definition applies to subsequent Test Runs on devices with the same model and OS version. Only one step per device model can be remediated in a single Test Run; additional fixes require a rerun. Once a remediation is submitted, the step cannot be re-resolved in the same session. + +== Live remediation +Live remediation allows users to resolve a blocker during an active Test Run without ending execution. It is available for Scriptless Test Runs. Contact your account manager or Kobiton Support to enable it for your organization. + +=== When live remediation is available + +Live remediation becomes available when a device in a Test Run encounters a blocker. At that moment, the device status changes from *Running* to *Live 5:00*, displayed with a stopwatch icon. The 5-minute countdown indicates how long live remediation remains available for that device. + +The timer runs independently per device. Execution pauses only for the affected device, and other devices in the same Test Run continue running. If the countdown expires before entry, the run follows standard blocker handling. + +image::live-remediation-countdown.png[width=600,alt="Test Run Results showing a device in the Live 5:00 state"] + +=== Entering a live remediation session + +Select the *Live 5:00* indicator on the affected device to open a manual session view. A banner appears above the device screen indicating that execution is paused, with an *Open KobiAI* option. + +Selecting *Open KobiAI* activates the Live Remediation tab in the right-side panel. This panel displays context about the blocked step and is where you interact with KobiAI to resolve the issue. + +=== Resolving the blocker with KobiAI + +Live remediation is driven entirely through the KobiAI chat panel. You cannot interact directly with the device screen during a live remediation session. Instead, you describe the resolution in natural language and KobiAI performs the interactions on the device. + +KobiAI describes what it detected, outlines the attempted action, and prompts you for confirmation or additional guidance. Respond directly in the panel to direct the next action. + +While KobiAI processes a submitted resolution, interaction with the device and chat panel is temporarily disabled. This is expected and indicates that the platform is completing the action. + +=== After the resolution + +When the resolution is applied, a confirmation banner appears and execution resumes from the paused step. The Test Run does not restart, and previously completed steps are not repeated. + +Actions submitted during live remediation update the Test Case definition. The update is saved as a new version and is reflected in Session Explorer. The updated definition applies to future Test Runs; previous runs remain unchanged. + +If a subsequent blocker is encountered later in the run, a new live remediation window begins with its own 5-minute countdown. + +=== If no resolution is submitted + +If you exit the session manually or allow the countdown to expire, the Test Run follows standard blocker handling and does not resume automatically. Remediation can still be completed afterward in Session Explorer. + +== Pop-up Handler remediation + +Pop-up Handler remediation helps handle dynamic or inconsistent pop-ups that appear during Scriptless Test Runs. Instead of failing when an unexpected screen appears, you can mark the pop-up as optional and define how to dismiss it without writing scripts. + +Use Pop-up Handler remediation when: + +* A pop-up appears during a revisit but was not present in the baseline session. +* A screen present in the baseline session does not appear during a revisit. +* Transient screens, such as onboarding prompts, survey modals, or OS alerts, interrupt test execution. + +Pop-up handling is defined during the session where the pop-up is encountered. After it is saved, the configuration applies to subsequent runs of the same Test Case on the same device model and OS. It does not apply retroactively to past sessions. + +=== Before you begin + +Before you use Pop-up Handler remediation: + +* An AI subscription is required. +* A baseline session must exist for the Test Case. +* The revisit must have encountered a blocker caused by a pop-up or unexpected screen. + +image::popup-handler.png[width=700,alt="Popup Handler tab in the Blocker Remediation window"] + +=== Handle a pop-up + +After a Test Run completes with a blocker caused by a pop-up, open the Blocker Remediation window. A detection message identifies which session contains the pop-up. + +. Select the *Popup Handler* tab. +. Under *Choose the screen with the pop-up*, select *Baseline* or *Revisit*. +. Enter hint text from the pop-up. The text must appear on the pop-up screen. +. Select *Highlight Text Area* to verify the hint text against the screenshot. ++ +Matched text is highlighted on the screenshot. If no match is found, adjust the hint text and try again. + +. Under *Choose the element to dismiss this pop-up*, search the view tree by path, label, or another attribute. +. Select the dismiss element. +. Select *Save as Optional*. + +== After remediation + +Remediation updates the test case definition. + +Once remediation is complete, rerunning the test validates that execution can proceed using the updated definition. Previous Test Runs remain unchanged and available for reference. + +== Related pages + +* xref:test-management:test-runs.adoc[Test Runs] +* xref:test-management:test-reruns.adoc[Test Reruns] +* xref:scriptless-automation:best-practices.adoc[Best practices] diff --git a/docs/modules/test-management/pages/test-cases.adoc b/docs/modules/test-management/pages/test-cases.adoc new file mode 100644 index 000000000..f32da05f0 --- /dev/null +++ b/docs/modules/test-management/pages/test-cases.adoc @@ -0,0 +1,140 @@ += Test Cases + +== Overview + +A Test Case represents a reusable test flow created from a baseline session. + +Test Cases store the interactions captured during a manual session and serve as the definition used for automated execution. Once created, a Test Case can be run repeatedly without recreating the original session. + +[#_convert_to_test_case] +== Convert a session to a Test Case + +Test Cases are created by converting completed manual sessions. + +After you run a manual session with capture enabled, Kobiton can convert the captured interactions into a reusable Test Case. + +You can convert a session to a Test Case from either of the following locations: + +* Session Explorer, after a completed manual session +* Test Management, by selecting *New Test Case* and choosing from the list of completed manual sessions + +In Test Management, completed manual sessions are listed from newest to oldest. Select the session you want to convert, then select *Convert*. + +In both cases, the Test Case is created from a captured manual session. The captured interactions from the baseline session define the test flow. + +Converting a manual session to a Test Case unlocks additional capabilities, including generating an Appium script and running the test across other devices. + +After conversion, the option to view the Test Case becomes available. Selecting this opens the Test Case details view, where the captured steps and interactions can be reviewed and managed. + +image::tm-newtcscreen.png[width=700,alt="Create a New Test Case in Test Management] + +== Manage Test Cases + +Test Cases are managed through Test Management in the Kobiton platform. + +=== Test Case list + +The Test Case list provides an overview of the Test Cases available to the selected Team. To view Test Cases for another Team, switch Teams. + +From this view, users can locate Test Cases, review basic information, and open individual Test Cases for detailed review and management. + +Use the search bar at the top of the list to find a specific Test Case by name or ID. + +The following actions are available for each Test Case in the list: + +* Select the Test Case ID to open the Test Case details view. Additional actions, such as generating an Appium script, are available there. +* *Run Test Case* — create a new Test Run from the existing definition. +* *Clone Test Case* — create a duplicate that can be modified independently. +* *Delete Test Case* — remove the Test Case from the list and from any test suites it was assigned to. + +To create a new Test Case from a past session, select *New Test Case* in the Test Cases view. Completed manual sessions that have not yet been converted are available for selection. + +=== Test Case details + +The Test Case details view displays the steps captured from the baseline session. + +From this view, users can: + +* Review captured test steps +* Remediate test steps when element selection requires correction +* Update Test Case metadata +* Assign the Test Case to one or more test suites +* Clone the Test Case +* Delete the Test Case +* Generate an Appium script + +Test Case details reflect the current version of the Test Case. + +==== Delete test steps + +Test steps can be deleted when they are no longer needed. + +. In the *Test Steps* section, select the *⋮* menu for the step you want to delete. +. Select *Delete Test Step* +. Repeat these steps for each step you want to delete. +. Select *Save*. + +Changes apply only after you select *Save*. Saved changes create a new version of the Test Case. Previous versions remain available from the *Test Case Version* dropdown. + +==== Edit text in test steps + +Test steps that use the *Enter Text* action support inline text editing. Use this option to update the text value entered during the step. + +. In the *Test Steps* section, hover over the *Enter Text* label for the step you want to update. +. In the preview window, select the edit icon. +. Update the text value. +. Select the checkmark icon to confirm the change. The edit icon changes to a checkmark while the text value is being edited. +. Select *Save*. + +Changes apply only after you select *Save*. Saved changes create a new version of the Test Case. + +image::tm-edit-text.png[width=700,alt="Enter text test step in edit mode with the context box open"] + +[#_remediate_test_steps] +==== Remediate test steps + +Test steps that use the *Touch* action can require remediation when Kobiton cannot reliably select the target element. Remediation is performed from Session Explorer and updates the Test Case definition. + +From the Test Case details page, you can open remediation for an individual touch step. In the step list, select the *⋮* menu next to the step, then select *Remediate Test Step*. If the step was previously remediated, select *Edit Remediation*. + +For the full procedure, see xref:test-management:remediations.adoc[Remediations]. + +After you save the remediation, Kobiton creates a new version of the Test Case. + +== Editing and versioning + +Changes to a Test Case create new versions. + +Versioning allows Test Cases to evolve as applications change while preserving previous definitions. Execution results remain associated with the version used at the time of execution. + +To view a previous version, open the Test Case details page, then select a version from the *Test Case Version* dropdown. + +Versions are listed chronologically, with the oldest version first and the newest version last. + +[NOTE] +==== +Changes to Test Case steps apply only after you select *Save*. Saving changes creates a new version of the Test Case. Previous versions remain available from the *Test Case Version* dropdown. +==== + +== Generate an Appium script + +An Appium script can be generated from a Test Case. + +From the Test Case details view, users can generate an Appium script based on the captured interactions in the Test Case. This script represents the same test flow and can be used as a starting point for script-based automation. + +=== When it’s used + +Generated Appium scripts are typically used when test definitions need to be extended, customized, or integrated into script-based automation workflows. + +Using Scriptless automation does not require generating or maintaining Appium scripts. + +If a Test Case is modified after an Appium script has been generated, the script must be regenerated to reflect the updated changes. + +For supported languages and frameworks, see xref:test-management:generate-appium.adoc[Generate an Appium script]. + +== Related pages + +* xref:scriptless-automation:run-scriptless.adoc[Run a Scriptless test] +* xref:scriptless-automation:baseline-session.adoc[Create a baseline session] +* xref:test-management:test-runs.adoc[Test runs] +* xref:test-management:test-suites.adoc[Test suites] diff --git a/docs/modules/test-management/pages/test-reruns.adoc b/docs/modules/test-management/pages/test-reruns.adoc new file mode 100644 index 000000000..d51717ac8 --- /dev/null +++ b/docs/modules/test-management/pages/test-reruns.adoc @@ -0,0 +1,41 @@ += Test Reruns + +== Overview + +A Test Rerun is a repeat execution of a previous Test Run using the same configuration. + +Reruns are used to validate results without redefining the test or changing execution settings. Each rerun is recorded as a separate execution and does not replace the original Test Run. + +== When Test Reruns are used + +Test Reruns are typically used after an issue has been addressed or when execution needs to be validated again. + +Common scenarios include: + +* Verifying execution after remediation +* Confirming behavior following an application update +* Repeating a Test Run to validate consistency + +Reruns ensure that results can be compared directly across executions. + +== How Test Reruns differ from new Test Runs + +A rerun repeats a previous Test Run using the same test definition, devices, and configuration. + +Creating a new Test Run allows changes to be made to the test setup, such as selecting different devices or modifying execution parameters. + +Both approaches create independent execution records, but reruns preserve the original execution context. + +== Where Test Reruns are managed + +Reruns are managed through Test Management and appear as separate Test Runs. + +Each rerun is listed with its own name and execution details. The rerun name includes a reference to the original Test Run ID, which can be used to identify the source of the rerun. + +Reruns can be reviewed individually to evaluate execution results after remediation or validation. + +== Related pages + +* xref:test-management:test-runs.adoc[Test Runs] +* xref:test-management:remediations.adoc[Remediations when execution requires user input] +* xref:test-management:test-cases.adoc[Test Cases] diff --git a/docs/modules/test-management/pages/test-runs.adoc b/docs/modules/test-management/pages/test-runs.adoc new file mode 100644 index 000000000..2660b0b77 --- /dev/null +++ b/docs/modules/test-management/pages/test-runs.adoc @@ -0,0 +1,159 @@ += Test Runs + +== Overview + +A Test Run represents a single execution of a Test Case or Test Suite on one or more devices. + +Test Runs apply an existing test definition to selected devices and record the outcome of that execution. Each Test Run is tracked independently and retains its own results and execution data. + +== Create a Test Run + +Test Runs are created from Test Management. + +A Test Run can be created using either a single Test Case or a Test Suite. During setup, users select the test definition, application version, and target devices. + +Once started, the platform executes the Test Cases automatically on the selected devices. + +The following fields are available on the Test Run Setup screen: + +[cols="1,3"] +|=== +| Field | Description + +| *Name* +| Required. A label for the Test Run. + +| *Description* +| Optional. Free-text context for the Test Run. + +| *Test Case* or *Test Suite* +| Required. Select one Test Case or one Test Suite to execute. Select *Customize* to choose a specific Test Case version, or to select and version the individual Test Cases within a suite. The latest version is selected by default. + +| *Application Versions* +| Appears after the Test Case or Test Suite is selected. If multiple application versions exist, a dropdown is shown. + +| *Device Bundle* or *Individual Devices* +| Required. See the device selection guidance below. + +| *Device Allocation Strategy* +| Applies only when running a Test Suite. Controls how Test Cases are distributed across the selected devices. Options are *All Permutations* (run each Test Case on each device) and *Random Allocation* (run each Test Case once on a randomly chosen device from the selection). +|=== + +Two options exist for selecting target devices, and they are mutually exclusive. + +*Device Bundle* runs the test on a pre-built group of devices. The *NOVA Recommended Devices* bundle is created by the system and is available by default. Custom bundles can be created from the device bundles area. Select *Customize* on a bundle to choose which devices in the bundle the test should target. + +*Individual Devices* allows selecting specific devices from a dropdown. Only online public and private devices assigned to the current Team appear in the list. Offline devices are not selectable. + +Test Runs can also be started directly from a Test Case or Test Suite: + +* From *Test Cases*, select the *Play* icon on a Test Case details page, or open the *⋮* menu on a Test Case in the list and choose *Run Test Case*. +* From *Test Suites*, open the *⋮* menu on a Test Suite in the list and choose *Run Test Suite*. + +In each case, the Test Run Setup screen opens with the Test Case or Test Suite already selected. + +image::test-run-setup.png[width=700,alt="Run Setup screen with all configuration fields visible"] + +== Manage Test Runs + +Test Runs are managed through Test Management in the platform. + +=== Test Run list + +The Test Run list displays all Test Runs available to the selected Team. From this view, users can review execution status, identify completed or in-progress runs, and access detailed results. + +Use the search field at the top of the list to find Test Runs by name, tags, OS, creator, or creation date. + +Select the Test Run *ID* to open the Test Run Results page. + +For each Test Run, select the *⋮* menu to access these actions: + +* *Rerun Test Run* — create a new Test Run with the same configuration. +* *Terminate Test Run* — stop all running revisit sessions for the run. +* *Delete Test Run* — remove the Test Run. + +image::test-run-list-actions.png[width=700,alt="Test Run list with the per-row action menu open showing Rerun, Terminate, and Delete"] + +=== Test Run details + +The Test Run Results page provides full information about a single Test Run, including configuration, execution progress, validation results, and access to individual sessions. + +The top of the page summarizes: + +* Application version, Test Run name, ID, and creator +* OS and number of devices in the run +* Created date and elapsed time +* Counts for *Crash* and *Accessibility* findings, which can be selected to filter the revisit executions grid + +Below the summary, a grid displays the status of each Test Case on each device. Statuses refresh automatically as execution progresses. See *Execution status* below for a list of statuses. + +Below each device's photo and model in the grid, a device-level status indicator displays the device's overall progress: *Action Completed*, *Action Needed*, or *Success*. + +The run is complete when every cell in the grid shows a non-*Running* status. + +To open the session for a specific cell, select its status. This navigates to the corresponding revisit session in Session Explorer, where execution details, screenshots, and step-by-step playback are available. + +Each Test Case row in the grid also displays a set of icons: + +* A lock icon, if the Test Case contains sensitive data. +* *View Test Case Details* — opens the Test Case Details page for the Test Case. +* *View Camera Roll* — opens the Camera Roll view for the Test Case. + +==== Automatic validations + +During execution, Kobiton automatically checks for *Crash* and *Accessibility* findings. The number of instances for each appears at the top of the Test Run Results page and can be selected to filter the grid to executions that contain those findings. + +==== Camera Roll + +Camera Roll is a visual comparison view that places the Test Case steps side-by-side with the steps actually executed during each Test Run session. It is used to review issues, confirm execution behavior, and access remediation when a blocker is encountered. + +To open Camera Roll, select the *View Camera Roll* icon on a Test Case row in the Test Run Results grid. + +Each device in Camera Roll is labeled with a status indicator: *Action Needed*, *Success*, or *Baseline*. + +In Camera Roll: + +* Expand an individual session to view the executed test steps. Only completed sessions can be expanded. +* If a blocker is encountered, select *Resolve Blocker* to remediate the affected step in Session Explorer. For the full remediation procedure, see xref:test-management:remediations.adoc[Remediations]. + +image::camera-roll-view.png[width=700,alt="Camera Roll view showing Test Case steps alongside executed steps"] + +== Execution status + +Each device executing a Test Case in a Test Run reports its own status. The Test Run Results page displays these statuses in a grid that updates as execution progresses. + +[cols="1,3"] +|=== +| Status | Meaning + +| *Running* +| The revisit execution has not started yet, or is in progress. + +| *Success* +| The revisit execution completed without error. + +| *Failed* +| The revisit execution failed because the app under test crashed. + +| *Need Action* +| The AI encountered a blocker and Blocker Remediation is required. + +| *Error* +| The revisit execution failed due to a Kobiton system issue. +|=== + +When a status appears with a fractional indicator, such as `4/4` or `2/3`, the fraction reflects the number of devices currently in that status out of the total number of devices in the run. + +== Rerun a Test Run + +Test Runs can be rerun using the same configuration. + +Reruns repeat the original execution with the same test definition, devices, and settings. This is commonly used after remediation or when validating execution behavior. + +Reruns do not replace the original Test Run. Each execution is a separate session and details can be found in Session Explorer. + +== Related pages + +* xref:test-management:test-cases.adoc[Test Cases] +* xref:test-management:test-suites.adoc[Test Suites] +* xref:test-management:remediations.adoc[Remediations] \ No newline at end of file diff --git a/docs/modules/test-management/pages/test-suites.adoc b/docs/modules/test-management/pages/test-suites.adoc new file mode 100644 index 000000000..fce21d31a --- /dev/null +++ b/docs/modules/test-management/pages/test-suites.adoc @@ -0,0 +1,46 @@ += Test Suites + +== Overview + +A Test Suite is a collection of related Test Cases grouped for organization and execution. + +Test Suites allow multiple Test Cases to be run together using a single configuration. They are commonly used to manage related test coverage and to scale execution across devices. + +== Create a Test Suite + +Test Suites are created in Test Management. + +To create a Test Suite, users define the operating system, provide a name and optional description, and select the Test Cases to include. Test Cases can be added to or removed from the suite as coverage evolves. + +Creating a Test Suite groups related Test Cases for execution but does not change the behavior of the individual Test Cases. Each Test Case retains its own definition and execution logic. + +After a Test Suite is created, it appears in the Test Suite list, where it can be run or managed. + +image::test-suite-list.png[width=700,alt="Create Test Suite"] + +== Manage Test Suites + +Test Suites are managed from the Test Suite list in Test Management. + +From the list, users can: + +* Run the Test Suite to create a Test Run +* Edit the suite name or description +* Update the set of Test Cases included +* Delete the Test Suite when it is no longer needed + +These actions are available from the suite’s action menu in the list view. + +== Run a Test Suite + +When a Test Suite is run, a configuration window is displayed before execution begins. + +In this step, users review and define execution settings such as the application version, devices or device bundles, and allocation strategy. These settings control how the Test Cases in the suite are distributed and executed. + +After the configuration is confirmed, the Test Suite is executed and a Test Run is created. + +== Related pages + +* xref:test-management:test-cases.adoc[Test Cases] +* xref:test-management:test-runs.adoc[Test Runs] +* xref:test-management:test-reruns.adoc[Test Reruns]