Fixed CT tests to run in OpenCHAMI by shunr-hpe · Pull Request #78 · OpenCHAMI/smd

shunr-hpe · 2025-10-07T20:44:12Z

Pull Request Template

Thank you for your contribution! Please ensure the following before submitting:

Checklist

My code follows the style guidelines of this project
I have added/updated comments where needed
I have added tests that prove my fix is effective or my feature works
I have run make test (or equivalent) locally and all tests pass
DCO Sign-off: All commits are signed off (git commit -s) with my real name and email
REUSE Compliance:
- Each new/modified source file has SPDX copyright and license headers
- Any non-commentable files include a <filename>.license sidecar
- All referenced licenses are present in the LICENSES/ directory

Description

Fixed CT tests to run in OpenCHAMI

Added Authorization header in the tavern tests
Added a script, smd-test, to run the tests.
Added an image for the tests.

Type of Change

Bug fix
New feature
Breaking change
Documentation update

For more info, see Contributing Guidelines.

- Added Authorization header in the tavern tests - Added a script, smd-test, to run the tests. - Added an image for the tests. Signed-off-by: Shane Unruh <shane.unruh@hpe.com>

README.md

- Changed smd-test to exit when a test fails and not continue on to the next set of tests - Updated the README to show running multiple test sets at once. Signed-off-by: Shane Unruh <shane.unruh@hpe.com>

alexlovelltroy · 2025-10-09T18:09:20Z

What does the output look like when running these tests?

shunr-hpe · 2025-10-09T23:20:25Z

The output looks like:

$ docker run -it --rm --network ${COMPOSE_NAME}_internal  smd-test:${SMD_VERSION}  smd-test test -t smoke
==================================================================== test session starts ====================================================================
platform linux -- Python 3.9.18, pytest-7.1.2, pluggy-1.6.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /
plugins: tavern-1.23.1, allure-pytest-2.12.0
collected 29 items                                                                                                                                          

opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/ready] PASSED                                                           [  3%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/liveness] PASSED                                                        [  6%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values] PASSED                                                          [ 10%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/arch] PASSED                                                     [ 13%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/class] PASSED                                                    [ 17%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/flag] PASSED                                                     [ 20%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/nettype] PASSED                                                  [ 24%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/role] PASSED                                                     [ 27%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/subrole] PASSED                                                  [ 31%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/state] PASSED                                                    [ 34%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/type] PASSED                                                     [ 37%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/State/Components] PASSED                                                        [ 41%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Defaults/NodeMaps] PASSED                                                       [ 44%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/DiscoveryStatus] PASSED                                               [ 48%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/Hardware] PASSED                                                      [ 51%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/HardwareByFRU] PASSED                                                 [ 55%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/Hardware/History] PASSED                                              [ 58%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/HardwareByFRU/History] PASSED                                         [ 62%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/RedfishEndpoints] PASSED                                              [ 65%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/ComponentEndpoints] PASSED                                            [ 68%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/ServiceEndpoints] PASSED                                              [ 72%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/EthernetInterfaces] PASSED                                            [ 75%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/groups] PASSED                                                                  [ 79%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/groups/labels] PASSED                                                           [ 82%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/partitions] PASSED                                                              [ 86%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/partitions/names] PASSED                                                        [ 89%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/memberships] PASSED                                                             [ 93%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Subscriptions/SCN] PASSED                                                       [ 96%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/sysinfo/powermaps] PASSED                                                       [100%]

==================================================================== 29 passed in 2.09s =====================================================================
$

shunr-hpe · 2025-10-09T23:23:53Z

The tests use pytest which can probably output in many different text formats.

For example, the smd-test script could run pytest with an option like:

pytest --junitxml=results.xml

I'll try that to see what happens.

shunr-hpe · 2025-10-09T23:30:43Z

My objective with this PR is to make SMD's existing tests minimally work.
I'm in favor of trying some other test frameworks, and seeing if any work better than the existing pytest with tavern framework.
However, I'm not in favor of throwing out the old before the new at least partially works.
And so my objective is to get the existing tests running but not spend too much time on it unless that is the long term direction.

shunr-hpe · 2025-10-10T00:21:29Z

Here is what a failure looks like when running the smoke tests

$ docker run -it --rm --network ${COMPOSE_NAME}_internal -v $(pwd)/test/ct:/tests/ct  smd-test:${SMD_VERSION}  smd-test test -t smoke
==================================================================== test session starts ====================================================================
platform linux -- Python 3.9.18, pytest-7.1.2, pluggy-1.6.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /
plugins: tavern-1.23.1, allure-pytest-2.12.0
collected 29 items                                                                                                                                          

opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/ready] PASSED                                                           [  3%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/liveness] PASSED                                                        [  6%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values] PASSED                                                          [ 10%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/arch] PASSED                                                     [ 13%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/class] PASSED                                                    [ 17%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/flag] PASSED                                                     [ 20%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/nettype] PASSED                                                  [ 24%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/role] FAILED                                                     [ 27%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/subrole] PASSED                                                  [ 31%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/state] PASSED                                                    [ 34%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/type] PASSED                                                     [ 37%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/State/Components] PASSED                                                        [ 41%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Defaults/NodeMaps] PASSED                                                       [ 44%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/DiscoveryStatus] PASSED                                               [ 48%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/Hardware] PASSED                                                      [ 51%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/HardwareByFRU] PASSED                                                 [ 55%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/Hardware/History] PASSED                                              [ 58%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/HardwareByFRU/History] PASSED                                         [ 62%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/RedfishEndpoints] PASSED                                              [ 65%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/ComponentEndpoints] PASSED                                            [ 68%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/ServiceEndpoints] PASSED                                              [ 72%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Inventory/EthernetInterfaces] PASSED                                            [ 75%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/groups] PASSED                                                                  [ 79%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/groups/labels] PASSED                                                           [ 82%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/partitions] PASSED                                                              [ 86%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/partitions/names] PASSED                                                        [ 89%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/memberships] PASSED                                                             [ 93%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/Subscriptions/SCN] PASSED                                                       [ 96%]
opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/sysinfo/powermaps] PASSED                                                       [100%]

========================================================================= FAILURES ==========================================================================
_____________________________________________________ test_smoke[Verify GET hsm/v2/service/values/role] _____________________________________________________

smoke_test_data = {'body': None, 'expected_status_code': 400, 'headers': {}, 'method': 'GET', ...}

    def test_smoke(smoke_test_data):
    
        headers=smoke_test_data["headers"]
        if smoke_test_data.get("use_auth"):
            if "Authorization" not in headers:
                access_token = get_access_token()
                headers["Authorization"] = f"Bearer {access_token}"
    
    
        req = requests.request(url=smoke_test_data["url"], method=str(smoke_test_data["method"]).upper(), data=smoke_test_data["body"],
                                       headers=smoke_test_data["headers"])
    
>       assert req.status_code == smoke_test_data["expected_status_code"], "unexpected status code."
E       AssertionError: unexpected status code.
E       assert 200 == 400
E        +  where 200 = <Response [200]>.status_code

opt/smd-test/smoke_pytest/test_smoke.py:25: AssertionError
================================================================== short test summary info ==================================================================
FAILED opt/smd-test/smoke_pytest/test_smoke.py::test_smoke[Verify GET hsm/v2/service/values/role] - AssertionError: unexpected status code.
=============================================================== 1 failed, 28 passed in 2.08s ================================================================
$
```

shunr-hpe · 2025-10-10T00:27:56Z

Here is a failure in the tavern tests

$ docker run -it --rm --network ${COMPOSE_NAME}_internal -v $(pwd)/test/ct:/tests/ct  smd-test:${SMD_VERSION}  smd-test test -t 2-non-disruptive
==================================================================== test session starts ====================================================================
platform linux -- Python 3.9.18, pytest-7.1.2, pluggy-1.6.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /
plugins: tavern-1.23.1, allure-pytest-2.12.0
collected 36 items                                                                                                                                          

tests/ct/api/2-non-disruptive/test_component_endpoints.tavern.yaml::Query the ComponentEndpoints collection PASSED                                    [  2%]
tests/ct/api/2-non-disruptive/test_component_endpoints.tavern.yaml::Query the ComponentEndpoints collection and get Node Information PASSED           [  5%]
tests/ct/api/2-non-disruptive/test_component_endpoints.tavern.yaml::Query the ComponentEndpoints collection and get NodeBMC Information PASSED        [  8%]
tests/ct/api/2-non-disruptive/test_component_endpoints.tavern.yaml::Query the ComponentEndpoints collection and get Node Enclosure Information PASSED [ 11%]
tests/ct/api/2-non-disruptive/test_component_endpoints.tavern.yaml::Attempt to query the ComponentEndpoints collection with a valid but nonexistent xname PASSED [ 13%]
tests/ct/api/2-non-disruptive/test_component_endpoints.tavern.yaml::Attempt to query the ComponentEndpoints collection with an invalid xname PASSED   [ 16%]
tests/ct/api/2-non-disruptive/test_component_endpoints.tavern.yaml::Attempt to query the ComponentEndpoints collection with a nid instead of an xname PASSED [ 19%]
tests/ct/api/2-non-disruptive/test_components.tavern.yaml::Verify that we can query for all Nodes in the Components collection PASSED                 [ 22%]
tests/ct/api/2-non-disruptive/test_components.tavern.yaml::Verify the Components collection PASSED                                                    [ 25%]
tests/ct/api/2-non-disruptive/test_components.tavern.yaml::Verify that we can query for all NodeBMCs in the Components collection PASSED              [ 27%]
tests/ct/api/2-non-disruptive/test_components.tavern.yaml::Verify that we can query for NodeBMCs in the Components collection FAILED                  [ 30%]
tests/ct/api/2-non-disruptive/test_components.tavern.yaml::Verify a variety of queries on the Components collection PASSED                            [ 33%]
tests/ct/api/2-non-disruptive/test_components.tavern.yaml::Verify the Component Query API with Node xnames PASSED                                     [ 36%]
tests/ct/api/2-non-disruptive/test_components.tavern.yaml::Verify that we can call the Component Query API with NodeBMC xnames PASSED                 [ 38%]
tests/ct/api/2-non-disruptive/test_discovery_status.tavern.yaml::Ensure that we can gather the system discovery status information PASSED             [ 41%]
tests/ct/api/2-non-disruptive/test_hardware.tavern.yaml::Query the Hardware collection PASSED                                                         [ 44%]
tests/ct/api/2-non-disruptive/test_hardware.tavern.yaml::Query the HardwareByFRU collection PASSED                                                    [ 47%]
tests/ct/api/2-non-disruptive/test_hardware.tavern.yaml::Query the Hardware collection for Node information PASSED                                    [ 50%]
tests/ct/api/2-non-disruptive/test_hardware.tavern.yaml::Query the Hardware collection for Node Enclosure information PASSED                          [ 52%]
tests/ct/api/2-non-disruptive/test_hardware.tavern.yaml::Call the Hardware Inventory Query API with a Node xname PASSED                               [ 55%]
tests/ct/api/2-non-disruptive/test_memberships.tavern.yaml::Ensure that we can gather information from the memberships collection PASSED              [ 58%]
tests/ct/api/2-non-disruptive/test_redfish_endpoints.tavern.yaml::Ensure that we can gather information from the RedfishEndpoints collection PASSED   [ 61%]
tests/ct/api/2-non-disruptive/test_service.tavern.yaml::hsmAPIservice PASSED                                                                          [ 63%]
tests/ct/api/2-non-disruptive/test_service_endpoints.tavern.yaml::Query the ServiceEndpoints collection PASSED                                        [ 66%]
tests/ct/api/2-non-disruptive/test_service_endpoints.tavern.yaml::Query the ServiceEndpoints collection for the AccountService PASSED                 [ 69%]
tests/ct/api/2-non-disruptive/test_service_endpoints.tavern.yaml::Query the ServiceEndpoints collection for the EventService PASSED                   [ 72%]
tests/ct/api/2-non-disruptive/test_service_endpoints.tavern.yaml::Query the ServiceEndpoints collection for the SessionService PASSED                 [ 75%]
tests/ct/api/2-non-disruptive/test_service_endpoints.tavern.yaml::Query the ServiceEndpoints collection for the UpdateService PASSED                  [ 77%]
tests/ct/api/2-non-disruptive/test_service_endpoints.tavern.yaml::Attempt to query the ServiceEndpoints collection for the AccountService with a valid but nonexistent xname PASSED [ 80%]
tests/ct/api/2-non-disruptive/test_service_endpoints.tavern.yaml::Attempt to query the ServiceEndpoints collection for the EventService with a valid but nonexistent xname PASSED [ 83%]
tests/ct/api/2-non-disruptive/test_service_endpoints.tavern.yaml::Attempt to query the ServiceEndpoints collection for the SessionService with a valid but nonexistent xname PASSED [ 86%]
tests/ct/api/2-non-disruptive/test_service_endpoints.tavern.yaml::Attempt to query the ServiceEndpoints collection for the UpdateService with a valid but nonexistent xname PASSED [ 88%]
tests/ct/api/2-non-disruptive/test_service_endpoints.tavern.yaml::Attempt to query the ServiceEndpoints collection for the AccountService with an invalid xname PASSED [ 91%]
tests/ct/api/2-non-disruptive/test_service_endpoints.tavern.yaml::Attempt to query the ServiceEndpoints collection for the EventService with an invalid xname PASSED [ 94%]
tests/ct/api/2-non-disruptive/test_service_endpoints.tavern.yaml::Attempt to query the ServiceEndpoints collection for the SessionService with an invalid xname PASSED [ 97%]
tests/ct/api/2-non-disruptive/test_service_endpoints.tavern.yaml::Attempt to query the ServiceEndpoints collection for the UpdateService with an invalid xname PASSED [100%]

========================================================================= FAILURES ==========================================================================
______________ /tests/ct/api/2-non-disruptive/test_components.tavern.yaml::Verify that we can query for NodeBMCs in the Components collection _______________
Format variables:
  hsm_base_url = 'http://smd:27779'
  xname = 'x0c0s1b0'
  verify = 'False'
  access_token = 'eyJhbGciOiJSUzI1NiIsImtpZCI6ImE1MzAxNTcyLWI1ODctNGZkMC1iM2RkLTI2OGMwZjg5NTYzNiIsInR5cCI6IkpXVCJ9.eyJhdWQiOltdLCJjbGllbnRfaWQiOiIyYjBlODBlMC04ZDQyLTRiNmItOGZmNC01M2Y2MTk0ZWQ5ZjMiLCJleHAiOjE3NjAwNTk0ODcsImV4dCI6e30sImlhdCI6MTc2MDA1NTg4NywiaXNzIjoiaHR0cHM6Ly9mb29iYXIub3BlbmNoYW1pLmNsdXN0ZXIvIiwianRpIjoiMjMxNzQwZmUtNTBmMi00MTQ0LTg5ZTctM2RlMjU1ZmU1NDcwIiwibmJmIjoxNzYwMDU1ODg3LCJzY3AiOlsic21kLnJlYWQiXSwic3ViIjoiMmIwZTgwZTAtOGQ0Mi00YjZiLThmZjQtNTNmNjE5NGVkOWYzIn0.ZdHUiis32yEUlGJlSnrf8MI7IJzkKrkvccm6Y3NsxO0m4-pdXDpdov1IjXnBU_92kPTDdY30JKORWLz-uw76SC4FOXH6JHgnV6SQG7wH6-fYObTVBzoccAk74oKS12Hr6thASBA-W9Rb9cBhvBVzrYTQOdaPGPCsTKKEWaZisckDkFdthIM94bv42knw_opW7PJp2Oe-wXaykh5Javs2vCeuZfaJPP0hb8cLt0-crZHMtCPYVkxUS1-bTFHXXZCPB2yvMVoDudt4H1nTDEm6tmEl7Du1xLMh8Q8ldzqQf_yKQ15gJ7Vmsa1vhMUcpMxeza5oEhX_Dk9j3UndoR6dD-Jiqhk0zQ3P7nFpJBgLaM1Gyy3uVSQ2WeBzVaHb-zwZ6vQb9V8gwLCBXI4J3Rd_v-qGwHGzHbpjNc0QGHX0OqmZiKcYqVJzmSwXmsbF7G5qD-2AgfszsB4-BfcGfo1vJ-vvaMooqzO4XUKjRitDwKCjyXpgfgt7h_DNcmXO4ARkMuEk3DdFugRbFh0VntxHt4KQnX4Dg5eplVPmxtFcrvrd-xIgt0SCwH0bXS32j_59tS01o704OWaWCUmMQYAR-r1QTNegNRP9ye0GDioJb8YxvLyy6Kr8swJ3QrtpIPCCfHbtdhMRZzEuEtOB5N773n9x2S_laVwfKkY_aNV68sc'
  xname = 'x0c0s1b0'

Source test stage (line 307):
  - name: Verify that we can query for a specific NodeBMC in the Components collection
    request:
      url: "{hsm_base_url}/hsm/v2/State/Components/{xname}"
      method: GET
      verify: !bool "{verify}"
      headers:
        Authorization: "Bearer {access_token}"
    response:
      status_code: 200
      verify_response_with:
        function: tavern.testutils.helpers:validate_pykwalify
        extra_kwargs:
          schema:
            type: map
            mapping:
              Arch:
                type: str
                required: True
              Class:
                type: str
                required: True
                enum:
                  - "fish"
                  - "rabbit"
                  - "dog"
              Enabled:
                type: bool
                required: True
              Flag:
                type: str
                required: True
              ID:
                type: str
                required: True
                enum:
                  - "{xname}"

              Locked:
                type: bool
                required: False

              Role:
                type: str
                required: False
              NetType:
                type: str
                required: True
              State:
                type: str
                required: True
              Type:
                type: str
                required: True
                enum:
                  - "NodeBMC"


Formatted stage:
  name: Verify that we can query for a specific NodeBMC in the Components collection
  request:
    headers:
      Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6ImE1MzAxNTcyLWI1ODctNGZkMC1iM2RkLTI2OGMwZjg5NTYzNiIsInR5cCI6IkpXVCJ9.eyJhdWQiOltdLCJjbGllbnRfaWQiOiIyYjBlODBlMC04ZDQyLTRiNmItOGZmNC01M2Y2MTk0ZWQ5ZjMiLCJleHAiOjE3NjAwNTk0ODcsImV4dCI6e30sImlhdCI6MTc2MDA1NTg4NywiaXNzIjoiaHR0cHM6Ly9mb29iYXIub3BlbmNoYW1pLmNsdXN0ZXIvIiwianRpIjoiMjMxNzQwZmUtNTBmMi00MTQ0LTg5ZTctM2RlMjU1ZmU1NDcwIiwibmJmIjoxNzYwMDU1ODg3LCJzY3AiOlsic21kLnJlYWQiXSwic3ViIjoiMmIwZTgwZTAtOGQ0Mi00YjZiLThmZjQtNTNmNjE5NGVkOWYzIn0.ZdHUiis32yEUlGJlSnrf8MI7IJzkKrkvccm6Y3NsxO0m4-pdXDpdov1IjXnBU_92kPTDdY30JKORWLz-uw76SC4FOXH6JHgnV6SQG7wH6-fYObTVBzoccAk74oKS12Hr6thASBA-W9Rb9cBhvBVzrYTQOdaPGPCsTKKEWaZisckDkFdthIM94bv42knw_opW7PJp2Oe-wXaykh5Javs2vCeuZfaJPP0hb8cLt0-crZHMtCPYVkxUS1-bTFHXXZCPB2yvMVoDudt4H1nTDEm6tmEl7Du1xLMh8Q8ldzqQf_yKQ15gJ7Vmsa1vhMUcpMxeza5oEhX_Dk9j3UndoR6dD-Jiqhk0zQ3P7nFpJBgLaM1Gyy3uVSQ2WeBzVaHb-zwZ6vQb9V8gwLCBXI4J3Rd_v-qGwHGzHbpjNc0QGHX0OqmZiKcYqVJzmSwXmsbF7G5qD-2AgfszsB4-BfcGfo1vJ-vvaMooqzO4XUKjRitDwKCjyXpgfgt7h_DNcmXO4ARkMuEk3DdFugRbFh0VntxHt4KQnX4Dg5eplVPmxtFcrvrd-xIgt0SCwH0bXS32j_59tS01o704OWaWCUmMQYAR-r1QTNegNRP9ye0GDioJb8YxvLyy6Kr8swJ3QrtpIPCCfHbtdhMRZzEuEtOB5N773n9x2S_laVwfKkY_aNV68sc
    method: GET
    url: 'http://smd:27779/hsm/v2/State/Components/x0c0s1b0'
    verify: !bool 'False'
  response:
    status_code: 200
    verify_response_with:
      extra_kwargs:
        schema:
          mapping:
            Arch:
              required: true
              type: str
            Class:
              enum:
              - fish
              - rabbit
              - dog
              required: true
              type: str
            Enabled:
              required: true
              type: bool
            Flag:
              required: true
              type: str
            ID:
              enum:
              - 'x0c0s1b0'
              required: true
              type: str
            Locked:
              required: false
              type: bool
            NetType:
              required: true
              type: str
            Role:
              required: false
              type: str
            State:
              required: true
              type: str
            Type:
              enum:
              - NodeBMC
              required: true
              type: str
          type: map
      function: tavern.testutils.helpers:validate_pykwalify

Errors:
E   tavern.util.exceptions.TestFailError: Test 'Verify that we can query for a specific NodeBMC in the Components collection' failed:
    - Error calling validate function '<function validate_pykwalify at 0x7fbd8420bee0>':
        Traceback (most recent call last):
          File "/usr/lib/python3.9/site-packages/tavern/schemas/files.py", line 106, in verify_generic
            verifier.validate()
          File "/usr/lib/python3.9/site-packages/pykwalify/core.py", line 194, in validate
            raise SchemaError(u"Schema validation failed:\n - {error_msg}.".format(
        pykwalify.errors.SchemaError: <SchemaError: error code 2: Schema validation failed:
         - Enum 'River' does not exist. Path: '/Class' Enum: ['fish', 'rabbit', 'dog'].: Path: '/'>
    
        The above exception was the direct cause of the following exception:
    
        Traceback (most recent call last):
          File "/usr/lib/python3.9/site-packages/tavern/response/base.py", line 152, in _maybe_run_validate_functions
            vf(response)
          File "/usr/lib/python3.9/site-packages/tavern/util/extfunctions.py", line 98, in inner
            result = func(response, *args, **kwargs)
          File "/usr/lib/python3.9/site-packages/tavern/testutils/helpers.py", line 103, in validate_pykwalify
            verify_generic(to_verify, schema)
          File "/usr/lib/python3.9/site-packages/tavern/schemas/files.py", line 109, in verify_generic
            raise BadSchemaError() from e
        tavern.util.exceptions.BadSchemaError
--------------------------------------------------------------------- Captured log call ---------------------------------------------------------------------
ERROR    pykwalify.core:core.py:190 validation.invalid
ERROR    pykwalify.core:core.py:191  --- All found errors ---
ERROR    pykwalify.core:core.py:192 ["Enum 'River' does not exist. Path: '/Class' Enum: ['fish', 'rabbit', 'dog']"]
ERROR    tavern.schemas.files:files.py:108 Error validating {'ID': 'x0c0s1b0', 'Type': 'NodeBMC', 'State': 'Ready', 'Flag': 'OK', 'Enabled': True, 'NetType': 'Sling', 'Arch': 'X86', 'Class': 'River'}
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/tavern/schemas/files.py", line 106, in verify_generic
    verifier.validate()
  File "/usr/lib/python3.9/site-packages/pykwalify/core.py", line 194, in validate
    raise SchemaError(u"Schema validation failed:\n - {error_msg}.".format(
pykwalify.errors.SchemaError: <SchemaError: error code 2: Schema validation failed:
 - Enum 'River' does not exist. Path: '/Class' Enum: ['fish', 'rabbit', 'dog'].: Path: '/'>
ERROR    tavern.response.base:base.py:41 Error calling validate function '<function validate_pykwalify at 0x7fbd8420bee0>':
    Traceback (most recent call last):
      File "/usr/lib/python3.9/site-packages/tavern/schemas/files.py", line 106, in verify_generic
        verifier.validate()
      File "/usr/lib/python3.9/site-packages/pykwalify/core.py", line 194, in validate
        raise SchemaError(u"Schema validation failed:\n - {error_msg}.".format(
    pykwalify.errors.SchemaError: <SchemaError: error code 2: Schema validation failed:
     - Enum 'River' does not exist. Path: '/Class' Enum: ['fish', 'rabbit', 'dog'].: Path: '/'>

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/usr/lib/python3.9/site-packages/tavern/response/base.py", line 152, in _maybe_run_validate_functions
        vf(response)
      File "/usr/lib/python3.9/site-packages/tavern/util/extfunctions.py", line 98, in inner
        result = func(response, *args, **kwargs)
      File "/usr/lib/python3.9/site-packages/tavern/testutils/helpers.py", line 103, in validate_pykwalify
        verify_generic(to_verify, schema)
      File "/usr/lib/python3.9/site-packages/tavern/schemas/files.py", line 109, in verify_generic
        raise BadSchemaError() from e
    tavern.util.exceptions.BadSchemaError
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/tavern/schemas/files.py", line 106, in verify_generic
    verifier.validate()
  File "/usr/lib/python3.9/site-packages/pykwalify/core.py", line 194, in validate
    raise SchemaError(u"Schema validation failed:\n - {error_msg}.".format(
pykwalify.errors.SchemaError: <SchemaError: error code 2: Schema validation failed:
 - Enum 'River' does not exist. Path: '/Class' Enum: ['fish', 'rabbit', 'dog'].: Path: '/'>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/tavern/response/base.py", line 152, in _maybe_run_validate_functions
    vf(response)
  File "/usr/lib/python3.9/site-packages/tavern/util/extfunctions.py", line 98, in inner
    result = func(response, *args, **kwargs)
  File "/usr/lib/python3.9/site-packages/tavern/testutils/helpers.py", line 103, in validate_pykwalify
    verify_generic(to_verify, schema)
  File "/usr/lib/python3.9/site-packages/tavern/schemas/files.py", line 109, in verify_generic
    raise BadSchemaError() from e
tavern.util.exceptions.BadSchemaError
===================================================================== warnings summary ======================================================================
usr/lib/python3.9/site-packages/_pytest/nodes.py:633
usr/lib/python3.9/site-packages/_pytest/nodes.py:633
usr/lib/python3.9/site-packages/_pytest/nodes.py:633
usr/lib/python3.9/site-packages/_pytest/nodes.py:633
usr/lib/python3.9/site-packages/_pytest/nodes.py:633
usr/lib/python3.9/site-packages/_pytest/nodes.py:633
usr/lib/python3.9/site-packages/_pytest/nodes.py:633
usr/lib/python3.9/site-packages/_pytest/nodes.py:633
  /usr/lib/python3.9/site-packages/_pytest/nodes.py:633: PytestRemovedIn8Warning: The (fspath: py.path.local) argument to YamlFile is deprecated. Please use the (path: pathlib.Path) argument instead.
  See https://docs.pytest.org/en/latest/deprecations.html#fspath-argument-for-node-constructors-replaced-with-pathlib-path
    return super().from_parent(parent=parent, fspath=fspath, path=path, **kw)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================== short test summary info ==================================================================
FAILED tests/ct/api/2-non-disruptive/test_components.tavern.yaml::Verify that we can query for NodeBMCs in the Components collection
========================================================= 1 failed, 35 passed, 8 warnings in 9.17s ==========================================================
$

shunr-hpe · 2025-10-10T00:30:29Z

The tavern test output is hard to read and this has made me not like tavern; however, I'd want to see the output from another framework before switching to it. i.e. all test output may be hard to read.

shunr-hpe · 2025-10-10T03:06:45Z

junitxml works (or could work if I updated the smd-test script). Tell me if there is a specific output you are interested in (text, or gui).

Here is the smoke test output in junit xml.

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
  <testsuite name="pytest" errors="0" failures="0" skipped="0" tests="29" time="2.279" timestamp="2025-10-10T02:58:24.006917" hostname="3d1f8cc512d7">
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/service/ready]" time="0.011"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/service/liveness]" time="0.012"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/service/values]" time="0.010"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/service/values/arch]" time="0.006"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/service/values/class]" time="0.006"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/service/values/flag]" time="0.011"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/service/values/nettype]" time="0.011"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/service/values/role]" time="0.010"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/service/values/subrole]" time="0.006"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/service/values/state]" time="0.007"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/service/values/type]" time="0.007"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/State/Components]" time="0.020"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/Defaults/NodeMaps]" time="0.120"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/Inventory/DiscoveryStatus]" time="0.114"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/Inventory/Hardware]" time="0.098"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/Inventory/HardwareByFRU]" time="0.117"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/Inventory/Hardware/History]" time="0.121"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/Inventory/HardwareByFRU/History]" time="0.102"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/Inventory/RedfishEndpoints]" time="0.146"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/Inventory/ComponentEndpoints]" time="0.146"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/Inventory/ServiceEndpoints]" time="0.119"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/Inventory/EthernetInterfaces]" time="0.025"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/groups]" time="0.154"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/groups/labels]" time="0.118"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/partitions]" time="0.107"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/partitions/names]" time="0.125"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/memberships]" time="0.112"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/Subscriptions/SCN]" time="0.132"/>
    <testcase classname="opt.smd-test.smoke_pytest.test_smoke" name="test_smoke[Verify GET hsm/v2/sysinfo/powermaps]" time="0.102"/>
  </testsuite>
</testsuites>

bmcdonald3

Thanks for addressing my concerns and showing output, looks good to me.

Fixed CT tests to run in OpenCHAMI

949ce35

- Added Authorization header in the tavern tests - Added a script, smd-test, to run the tests. - Added an image for the tests. Signed-off-by: Shane Unruh <shane.unruh@hpe.com>

shunr-hpe requested a review from bmcdonald3 October 7, 2025 20:44

bmcdonald3 reviewed Oct 7, 2025

View reviewed changes

README.md Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

Changed smd-test to better support running multiple test sets

e225538

- Changed smd-test to exit when a test fails and not continue on to the next set of tests - Updated the README to show running multiple test sets at once. Signed-off-by: Shane Unruh <shane.unruh@hpe.com>

bmcdonald3 approved these changes Oct 10, 2025

View reviewed changes

shunr-hpe merged commit fa1727c into main Oct 15, 2025
7 checks passed

shunr-hpe deleted the ct-tests branch October 15, 2025 18:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixed CT tests to run in OpenCHAMI#78

Fixed CT tests to run in OpenCHAMI#78
shunr-hpe merged 2 commits intomainfrom
ct-tests

shunr-hpe commented Oct 7, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

alexlovelltroy commented Oct 9, 2025

Uh oh!

shunr-hpe commented Oct 9, 2025

Uh oh!

shunr-hpe commented Oct 9, 2025

Uh oh!

shunr-hpe commented Oct 9, 2025

Uh oh!

shunr-hpe commented Oct 10, 2025

Uh oh!

shunr-hpe commented Oct 10, 2025

Uh oh!

shunr-hpe commented Oct 10, 2025 •

edited

Loading

Uh oh!

shunr-hpe commented Oct 10, 2025

Uh oh!

bmcdonald3 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

shunr-hpe commented Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Template

Checklist

Description

Type of Change

Uh oh!

Uh oh!

Uh oh!

alexlovelltroy commented Oct 9, 2025

Uh oh!

shunr-hpe commented Oct 9, 2025

Uh oh!

shunr-hpe commented Oct 9, 2025

Uh oh!

shunr-hpe commented Oct 9, 2025

Uh oh!

shunr-hpe commented Oct 10, 2025

Uh oh!

shunr-hpe commented Oct 10, 2025

Uh oh!

shunr-hpe commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shunr-hpe commented Oct 10, 2025

Uh oh!

bmcdonald3 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shunr-hpe commented Oct 7, 2025 •

edited

Loading

shunr-hpe commented Oct 10, 2025 •

edited

Loading