Add a description to the SystemReady Devicetree categories#635
Add a description to the SystemReady Devicetree categories#635vstehle wants to merge 2 commits intoARM-software:mainfrom
Conversation
Add a JSON schema describing the format of test_categoryDT.json.
The following example command can be used for validation, from the
common/log_parser folder:
$ check-jsonschema --schemafile test_categoryDT-schema.json \
test_categoryDT.json
Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>
Add a `Description' field to each test suite in test_categoryDT.json. This is meant to help test engineers better understand the implication of each test suite's failures on system quality. The JSON format does not allow to break long lines. Therefore, to keep the file human-readable, the description is recorded as an array of strings, with one string corresponding to one sentence. While at it, update the schema. Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>
|
Hi @chetan-rathore and @pergar01, could you please have a look? Test suites are never perfectly documented regarding what they test exactly. Therefore, feel free to add even more details. |
There was a problem hiding this comment.
Thanks @vstehle for adding the schema. The schema should be placed in common/tools directory.
| "The Base Boot Requirements (BBR) Platform Design Document specifies requirements for boot and runtime services that operating systems and hypervisors can rely on.", | ||
| "The SystemReady Devicetree band recommends compliance with BBR requirements on SMBIOS data structures.", | ||
| "Failures are not expected to prevent an OS from booting.", | ||
| "https://github.com/ARM-software/arm-systemready/blob/main/common/log_parser/standalone_tests/logs_to_json.py" |
There was a problem hiding this comment.
Since SMBIOS test is run as part of SCT, this link should point to https://github.com/tianocore/edk2-test
| "Description": [ | ||
| "The SystemReady Devicetree band recommends network boot support.", | ||
| "Failures may have only minor effects, or may prevent an OS to boot from the network.", | ||
| "https://github.com/ARM-software/arm-systemready/blob/main/common/log_parser/standalone_tests/logs_to_json.py" |
There was a problem hiding this comment.
Link should point to the test script: https://github.com/ARM-software/arm-systemready/blob/main/SystemReady-devicetree-band/Yocto/meta-woden/recipes-acs/install-files/files/ethtool-test.py
| "Description": [ | ||
| "The SystemReady Devicetree band defines requirements on the Devicetree presented to an OS.", | ||
| "Failures may have only minor effects, or may prevent an OS to boot.", | ||
| "https://github.com/ARM-software/arm-systemready/blob/main/common/log_parser/standalone_tests/logs_to_json.py" |
There was a problem hiding this comment.
As part of Peripherals standalone, the linux DT kselftest is run..we can point to https://docs.kernel.org/dev-tools/testing-devices.html
| "Description": [ | ||
| "The SystemReady Devicetree band recommends support for A/B firmware update.", | ||
| "Failures may have only minor effects, or may prevent reliable firmware update.", | ||
| "https://github.com/ARM-software/arm-systemready/blob/main/common/log_parser/standalone_tests/logs_to_json.py" |
There was a problem hiding this comment.
This is not tested by ACS at this point, we can specify that
| "The EBBR specification defines a subset of UEFI provided to an OS.", | ||
| "This comprises requirements when firmware is updated in-band.", | ||
| "Failures may have only minor effects, or may prevent firmware update.", | ||
| "https://github.com/ARM-software/arm-systemready/blob/main/common/log_parser/standalone_tests/logs_to_json.py" |
There was a problem hiding this comment.
Link can point to capsule update testing https://github.com/ARM-software/arm-systemready/blob/52cb4e39c1b7ff700a7909bacb6ed4de9cc8d8ab/docs/Automatic_Capsule_Update_guide.md
chetan-rathore
left a comment
chetan-rathore
left a comment
There was a problem hiding this comment.
Thanks Vincent for adding description for each test category. There are some minor comments from acs side, if its fine with you ACS can make the changes on top of the PR
2 participants
This is done in two steps:
test_categoryDT.jsonas it isUltimately those descriptions should end up into the tests results JSON and HTML files, to be visible by test engineers.
The schema is not applicable to
test_category.jsonas the formats differ.