Add kolega.dev scan results for Benchmark v1.2#426
Add kolega.dev scan results for Benchmark v1.2#426kolega-ai-dev wants to merge 1 commit intoOWASP-Benchmark:masterfrom
Conversation
kolega.dev is an AI-agent-based security analysis platform. Results are in Semgrep JSON format, compatible with the existing SemgrepReader in BenchmarkUtils. Scores: TPR 96.9%, FPR 9.5%, Youden's Index +87.4%
|
@darkspirit510 - Sascha, can you check this out, verify the results? And for our regression testing we need a results file we can remove most of the results of. Per the PR, you can request a trial account to use their tool. |
|
@davewichers this is just a result file. I'm not sure if we should add commercial results to the repo (even though it was provided by the author(s)) @kolega-ai-dev sorry for the late reply, got too many stuff on my desk. Even though you mentioned this is Semgrep reader compatible, I prefer separate readers for each tool. That way it's clear what Benchmark is able to read. If you want, I can create a reader based on the provided result file. |
|
@kolega-ai-dev - yes, please do NOT add any full commercial results files or scorecards to BenchmarkJava. You can add snippets for regression testing to BenchmarkUtils, but that's it. So, I'm closing this. |
Ref: Issue 427
[Kolega.dev](https://kolega.dev) is an AI-agent-based security analysis platform. Results are in Semgrep JSON format, compatible with the existing SemgrepReader in BenchmarkUtils.
Scores: TPR 96.9%, FPR 9.5%, Youden's Index +87.4%
Generated scorecard: https://kolega-public-artifacts.s3.amazonaws.com/benchmarks/owasp_java_v1.2/scorecard/Benchmark_v1.2_Scorecard_for_Semgrep_v1.0.html
How to reproduce:
Contact faizan@kolega.ai for access to the Kolega scanner. We can provision a trial account to verify these results independently.
Our tool is commercial. Happy for this to be included as an anonymised entry if named inclusion isn't possible.