Skip to content

Latest commit

 

History

History
92 lines (72 loc) · 3.34 KB

performance.md

File metadata and controls

92 lines (72 loc) · 3.34 KB

KICS Accuracy Benchmark

The values below were obtained after scanning 150 open source projects with KICS (1.2.x) covering the supported IaC technologies (c.f., Terraform, Ansible, Kubernetes, Docker, AWS Cloudformation).

IaC Technology Query Accuracy1 Query Coverage2 Scanned IaC files​ Number of Results​ Average Scan Time​ (s) Average Project Size (MB)
Terraform​ 99.7%​ 46% 1176​ 709 6.6 33.4​
Docker​ 98.8%​​ 68%​ 1017​ 5109 11 0.7
Kubernetes​ 99.3%​​ 88.7%​ 6089​ 21753 7 90
CloudFormation​ 95%​ 73%​ 1769​ 5343 10.2 4.8
Ansible ​ 100% ​ 54%​ 3367​ 1320 23.3 4.1

1 Query Accuracy = TP results / results

2 Query Coverage = Query with results / Queries


Global Measures

Measure Value
Average Accuracy 98.6%
Total Number of Results 34234
Average Query Coverage 66.4%
Total Scanned IaC Files 13418
Average Scan Time (s) 11.2
Average Project Size (MB) 26.6

KICS Profiling

Running Kics with --profiling flag will log the CPU/MEM metrics used for:

  • Getting Queries
  • Parsing Files
  • Scanning Vulnerabilities
  • Generating Reports

Keep in mind that profiling will periodically stop KICS to retrieve the wanted metrics, meaning KICS execution time will increase substantially.

CPU Profiling

Flag: --profiling CPU

9:43AM INF Scanning with Keeping Infrastructure as Code Secure dev
9:43AM INF Total CPU usage for get_queries: 6.56s <-
9:43AM INF Inspector initialized, number of queries=1385
9:43AM INF Total CPU usage for get_sources: 200.00ms <-
9:43AM INF Total CPU usage for inspect: 15.43s <-
9:43AM INF Results saved to file results/results.json fileName=results.json
9:43AM INF Results saved to file results/results.sarif fileName=results.sarif
9:43AM INF Results saved to file results/results.html fileName=results.html
9:43AM INF Total CPU usage for generate_report: 290.00ms <-
9:43AM INF Files scanned: 221
9:43AM INF Parsed files: 221
9:43AM INF Queries loaded: 1385
9:43AM INF Queries failed to execute: 0
9:43AM INF Inspector stopped
9:43AM INF Scan duration: 21.1476197s

MEM Profiling

Flag: --profiling MEM

9:43AM INF Scanning with Keeping Infrastructure as Code Secure dev
9:43AM INF Total MEM usage for get_queries: 237.96MB <-
9:43AM INF Inspector initialized, number of queries=1385
9:43AM INF Total MEM usage for get_sources: 280.53MB <-
9:43AM INF Total MEM usage for inspect: 335.44MB <-
9:43AM INF Results saved to file results/results.json fileName=results.json
9:43AM INF Results saved to file results/results.sarif fileName=results.sarif
9:43AM INF Results saved to file results/results.html fileName=results.html
9:43AM INF Total MEM usage for generate_report: 333.38MB <-
9:43AM INF Files scanned: 221
9:43AM INF Parsed files: 221
9:43AM INF Queries loaded: 1385
9:43AM INF Queries failed to execute: 0
9:43AM INF Inspector stopped
9:43AM INF Scan duration: 21.1476197s