-
Notifications
You must be signed in to change notification settings - Fork 358
feat(action): added validation for searchLine field in actions #7994
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
cx-ricardo-jesus
wants to merge
71
commits into
master
Choose a base branch
from
AST-139912--searchLine-validation
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+231
−6
Draft
Changes from all commits
Commits
Show all changes
71 commits
Select commit
Hold shift + click to select a range
a0ef30f
changed go-ci.yml to use a script to check searchLine
cx-ricardo-jesus bd0d688
.
cx-ricardo-jesus 823d88c
fixed print to not use f when is missing placeholders
cx-ricardo-jesus 6d64470
removed unnecessary action
cx-ricardo-jesus b8986d7
fixed typo
cx-ricardo-jesus 8040087
fixing issues from codacy
cx-ricardo-jesus 2b517c5
updated go image in Dockerfile
cx-ricardo-jesus e18bee3
update go images
cx-ricardo-jesus 2312cda
changed git image
cx-ricardo-jesus 09a2cb0
changing positive expected results
cx-ricardo-jesus a64ba82
added requests to requirements file
cx-ricardo-jesus cc2b2ef
changed searchLine to get -1 value
cx-ricardo-jesus 2c99993
added debug prints
cx-ricardo-jesus e46f8ff
added exception type
cx-ricardo-jesus 8a6e905
fixing error in script
cx-ricardo-jesus b869010
removed trailing whitespace
cx-ricardo-jesus 989adfe
changed to run the other script
cx-ricardo-jesus cdb60a5
debugging test directory path
cx-ricardo-jesus d37f8c5
changed scripts that run on the action
cx-ricardo-jesus 5ee1892
removed f-string without placeholders
cx-ricardo-jesus ee9a137
inverted changes on the query
cx-ricardo-jesus 8ed8261
changed positive_expected_result
cx-ricardo-jesus 4e92c36
changed filename to fileName
cx-ricardo-jesus c1942b4
added print for debugging processes
cx-ricardo-jesus f714b7e
added print to see content value
cx-ricardo-jesus 6a74250
changed script
cx-ricardo-jesus dcff885
testing searchLine != searchLine
cx-ricardo-jesus 4d48c0e
removed unnecessary sorting on the results in execution context
cx-ricardo-jesus b3d85ab
removed unused requirements.txt file
cx-ricardo-jesus c374d8d
reverter changes on positie_expected_results
cx-ricardo-jesus 6606dbb
Merge branch 'master' into AST-139912--searchLine-validation
cx-ricardo-jesus 63e3406
Merge branch 'master' into AST-139912--searchLine-validation
cx-ricardo-jesus d5e7348
changes on the script and removed unnecessary go setup
cx-ricardo-jesus ded7ad2
changed query to get errors on go-ci action
cx-ricardo-jesus bf3939b
parsing json data into objects
cx-ricardo-jesus 135d8f9
small change to test if this is the root of the problem
cx-ricardo-jesus cf22572
reverted changes
cx-ricardo-jesus 2bef2ed
fixing misspelled directory name
cx-ricardo-jesus 4475b51
fixing some errors in the script used in the validate-search-line job…
cx-ricardo-jesus 78bad5f
fixed misspelled field in validate_scan_results function
cx-ricardo-jesus bd289fd
simplifying code
cx-ricardo-jesus 972a039
using pymarshal to use models that unmarshal json content directly
cx-ricardo-jesus 1576282
fixing errors in results_models pymarshal import
cx-ricardo-jesus 581fa12
testing searchLine defined to '-1' hardcoded value
cx-ricardo-jesus 0504c8b
convert line and searchLine to int
cx-ricardo-jesus df27dd2
adding print's for debug purposes
cx-ricardo-jesus 1a1cf97
adding print's for debug purposes
cx-ricardo-jesus 0ece437
fixing cases where searchLine is defined to an hardcoded value of -1
cx-ricardo-jesus 7d6d715
added prints for debug purposes
cx-ricardo-jesus 28c3dff
debug prints
cx-ricardo-jesus 290acf3
debug prints
cx-ricardo-jesus d0dab72
debug prints
cx-ricardo-jesus 1416d96
changes for debug purposes
cx-ricardo-jesus 216c56f
.
cx-ricardo-jesus f3038c4
changes
cx-ricardo-jesus 01ebac7
trying to fix cases with -1 hardcoded on searchLine
cx-ricardo-jesus be3f72c
changes
cx-ricardo-jesus 411f4f3
more changes
cx-ricardo-jesus c623658
trying type_assert
cx-ricardo-jesus 51eb564
trying type_assert
cx-ricardo-jesus 27b1798
trying type_assert
cx-ricardo-jesus a20d193
trying type_assert
cx-ricardo-jesus f56d8cf
trying type_assert
cx-ricardo-jesus 39f673e
testing new scenario
cx-ricardo-jesus e975f6c
testing test3 case from artur test branch
cx-ricardo-jesus ef20b51
testing test3 case from artur test branch
cx-ricardo-jesus b29765c
back to the other scenario
cx-ricardo-jesus 3cfcdf4
back to the other scenario again
cx-ricardo-jesus 4f2b0ab
used unmarshal_json from pumarshal to unmarshal the json with the res…
cx-ricardo-jesus afbd895
testing hardcoded .1 again
cx-ricardo-jesus 496648e
trying case - common_lib.build_search_line([shdfosdhfoisdhf], []),
cx-ricardo-jesus File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| from pymarshal.json import type_assert, type_assert_iter | ||
|
|
||
|
|
||
| class ScanFile: | ||
| def __init__(self, file_name, similarity_id, line, | ||
| resource_type, resource_name, issue_type, | ||
| search_key, search_line, search_value, | ||
| expected_value, actual_value): | ||
| self.file_name = type_assert(file_name, str) | ||
| self.similarity_id = type_assert(similarity_id, str) | ||
| self.line = type_assert(line, int) | ||
| self.resource_type = type_assert(resource_type, str) | ||
| self.resource_name = type_assert(resource_name, str) | ||
| self.issue_type = type_assert(issue_type, str) | ||
| self.search_key = type_assert(search_key, str) | ||
| self.search_line = type_assert(search_line, int) | ||
| self.search_value = type_assert(search_value, str) | ||
| self.expected_value = type_assert(expected_value, str) | ||
| self.actual_value = type_assert(actual_value, str) | ||
|
|
||
|
|
||
| class Query: | ||
| def __init__(self, query_name="", query_id="", files=None): | ||
| self.query_name = type_assert(query_name, str) | ||
| self.query_id = type_assert(query_id, str) | ||
| self.files = type_assert_iter(files, ScanFile) | ||
|
|
||
|
|
||
| class ScanResults: | ||
| def __init__(self, queries=None): | ||
| self.queries = type_assert_iter(queries, Query) |
160 changes: 160 additions & 0 deletions
160
.github/scripts/validate-search-line/validate_search_line.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,160 @@ | ||
| #!/usr/bin/env python3 | ||
|
|
||
| import json | ||
| import os | ||
| import subprocess | ||
| import sys | ||
| from pathlib import Path | ||
| from pymarshal.json import unmarshal_json | ||
| from results_models import ScanResults | ||
|
|
||
| REPO_ROOT = Path(__file__).resolve().parents[3] | ||
|
|
||
| def get_changed_queries(): | ||
| """Parse CHANGED_QUERIES env var (JSON array from dorny/paths-filter) to get query directories.""" | ||
| raw = os.getenv("CHANGED_QUERIES", "") | ||
| if not raw: | ||
| print("::error::CHANGED_QUERIES environment variable is empty or not set") | ||
| sys.exit(1) | ||
|
|
||
| try: | ||
| files = json.loads(raw) | ||
| except json.JSONDecodeError: | ||
| print(f"::error::CHANGED_QUERIES is not valid JSON: {raw}") | ||
| sys.exit(1) | ||
|
|
||
| dirs = [] | ||
| for f in files: | ||
| if f.endswith("/query.rego"): | ||
| dirs.append(REPO_ROOT / Path(f).parent) | ||
| return dirs | ||
|
|
||
|
|
||
| def has_search_line_defined(query_dir): | ||
| """Check if query.rego defines searchLine in its result object.""" | ||
| rego_file = query_dir / "query.rego" | ||
| if not rego_file.exists(): | ||
| return False | ||
| return "searchLine" in rego_file.read_text() | ||
|
|
||
|
|
||
| def run_kics_scan(query_dir): | ||
| """Run KICS scan for a single query and return True if it completed successfully.""" | ||
| query_id = json.loads((query_dir / "metadata.json").read_text())["id"] | ||
|
|
||
| results_dir = query_dir / "results" | ||
| results_dir.mkdir(exist_ok=True) | ||
|
|
||
| payloads_dir = query_dir / "payloads" | ||
| payloads_dir.mkdir(exist_ok=True) | ||
|
|
||
| cmd = [ | ||
| "go", "run", str(REPO_ROOT / "cmd" / "console" / "main.go"), | ||
| "scan", | ||
| "-p", str(query_dir / "test"), | ||
| "-o", str(results_dir), | ||
| "--output-name", "all_results.json", | ||
| "-i", query_id, | ||
| "-d", str(payloads_dir / "all_payloads.json"), | ||
| "-v", | ||
| "--experimental-queries", | ||
| "--bom", | ||
| "--enable-openapi-refs", | ||
| "--ignore-on-exit", "results", | ||
cx-ricardo-jesus marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| "--kics_compute_new_simid" | ||
| ] | ||
|
|
||
| print(f" Running scan with query ID: {query_id}") | ||
|
|
||
| proc = subprocess.run(cmd, capture_output=True, text=True, cwd=str(REPO_ROOT)) | ||
|
|
||
| if proc.returncode not in {0, 60, 50, 40, 30, 20}: | ||
| print(f" ::error::Scan failed (exit code {proc.returncode})") | ||
| if proc.stdout: | ||
| print(f" stdout (last 500 chars): ...{proc.stdout[-500:]}") | ||
| if proc.stderr: | ||
| print(f" stderr (last 500 chars): ...{proc.stderr[-500:]}") | ||
| return False | ||
|
|
||
| return True | ||
|
|
||
| def validate_scan_results(query_dir): | ||
| """ | ||
| Validate scan results: | ||
| - Fail if any search_line != line | ||
| - Fail if any search_line == -1 | ||
| """ | ||
| results_file = query_dir / "results" / "all_results.json" | ||
| rel_dir = query_dir.relative_to(REPO_ROOT) | ||
|
|
||
| if not results_file.exists(): | ||
| print(f" ::error file={rel_dir}::Results file not generated by scan") | ||
| return False | ||
|
|
||
| data = json.loads(results_file.read_text()) | ||
| scan_results = unmarshal_json(data, ScanResults) | ||
|
|
||
| # Flatten results from all queries | ||
| all_results = [] | ||
| for query in scan_results.queries: | ||
| all_results.extend(query.files) | ||
|
|
||
| if not all_results: | ||
| print(" [OK] No results to validate") | ||
| return True | ||
| # Validate each result | ||
| valid = True | ||
| for idx, f in enumerate(all_results): | ||
| sl = int(f.search_line) | ||
| ln = int(f.line) | ||
| fn = f.file_name | ||
|
|
||
| if sl == -1: | ||
| print(f" ::error::Result [{idx}] {fn}: search_line is -1") | ||
| valid = False | ||
| elif sl != ln: | ||
cx-ricardo-jesus marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| print(f" ::error::Result [{idx}] {fn}: search_line ({sl}) != line ({ln})") | ||
| valid = False | ||
| else: | ||
| print(f" [OK] Result [{idx}] {fn}: search_line={sl} == line={ln}") | ||
|
|
||
| return valid | ||
|
|
||
|
|
||
| def validate_query(query_dir): | ||
| """Validate a single query directory.""" | ||
|
|
||
| if not has_search_line_defined(query_dir): | ||
| print(" [SKIP] searchLine not defined in query.rego - PASS") | ||
| return True | ||
|
|
||
| print(" searchLine is defined in query.rego - running scan...") | ||
|
|
||
| if not run_kics_scan(query_dir): | ||
| return False | ||
|
|
||
| return validate_scan_results(query_dir) | ||
|
|
||
|
|
||
| def main(): | ||
| query_dirs = get_changed_queries() | ||
|
|
||
| if not query_dirs: | ||
| print("No query.rego were changed - nothing to validate") | ||
| sys.exit(0) | ||
|
|
||
| all_valid = True | ||
| for qd in query_dirs: | ||
| if not validate_query(qd): | ||
| all_valid = False | ||
|
|
||
| if all_valid: | ||
| print("All searchLine validations passed!") | ||
| sys.exit(0) | ||
| else: | ||
| print("::error::Some searchLine validations failed. See errors above.") | ||
| sys.exit(1) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why change the dockerfiles go version? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.