Merge pull request Checkmarx#4463 from Checkmarx/docs/add-bom-docs-up…

…date-command-flags docs: add BoM, update flags
Dipesh-Reen · Nov 3, 2021 · a8d746d · a8d746d
2 parents e245d32 + aaf943b
commit a8d746d
Show file tree

Hide file tree

Showing 4 changed files with 139 additions and 47 deletions.
diff --git a/docs/bom.md b/docs/bom.md
@@ -0,0 +1,87 @@
+## Bill Of Materials
+
+This feature uses Rego queries to extract a list of used Terraform resources along with its metadata in the scanned IaC.
+
+Find the existing queries under: [assets/queries/terraform/aws_bom](https://github.com/Checkmarx/kics/tree/master/assets/queries/terraform/aws_bom)
+
+BoM queries extracts metadata about the resources and organizes it in the following structure:
+
+```go
+billOfMaterialsRequiredFields := map[string]bool{
+    "resource_type":          true,
+    "resource_name":          true,
+    "resource_engine":        false,
+    "resource_accessibility": true,
+    "resource_vendor":        true,
+    "resource_category":      true,
+}
+```
+
+After extracting the information the query stores the stringified JSON structure inside the `value` field in the `result`:
+
+```rego
+CxPolicy[result] {
+	bucket_resource := input.document[i].resource.aws_s3_bucket[name]
+
+	bom_output = {
+		"resource_type": "aws_s3_bucket",
+		"resource_name": get_bucket_name(bucket_resource),
+		"resource_accessibility": get_accessibility(bucket_resource, name),
+		"resource_vendor": "AWS",
+		"resource_category": "Storage",
+	}
+
+	result := {
+		"documentId": input.document[i].id,
+		"searchKey": sprintf("aws_s3_bucket[%s]", [name]),
+		"issueType": "BillOfMaterials",
+		"keyExpectedValue": "",
+		"keyActualValue": "",
+		"searchLine": common_lib.build_search_line(["resource", "aws_s3_bucket", name], []),
+		"value": json.marshal(bom_output),
+	}
+}
+```
+
+### Results
+
+Results will be found in the [JSON](results.md) output and placed separately under `bill_of_materials` key:
+
+```json
+{
+    // etc...
+    "bill_of_materials": [
+        {
+            "query_name": "BOM - MSK",
+            "query_id": "051f2063-2517-4295-ad8e-ba88c1bf5cfc",
+            "query_url": "https://kics.io",
+            "severity": "TRACE",
+            "platform": "Terraform",
+            "category": "Bill Of Materials",
+            "description": "A list of MSK resources specified. Amazon Managed Streaming for Apache Kafka (Amazon MSK) is a fully managed service that enables you to build and run applications that use Apache Kafka to process streaming data.",
+            "description_id": "cf7ae008",
+            "files": [
+                {
+                    "file_name": "sample1.tf",
+                    "similarity_id": "9c1bd86b2367fd748ed66a86f72d637231be6e4ec04d68dd10a61f233187b777",
+                    "line": 1,
+                    "issue_type": "BillOfMaterials",
+                    "search_key": "aws_msk_cluster[example_msk1]",
+                    "search_line": 0,
+                    "search_value": "",
+                    "expected_value": "",
+                    "actual_value": "",
+                    "value": "{\"resource_accessibility\":\"encrypted\",\"resource_category\":\"Queues\",\"resource_name\":\"example\",\"resource_type\":\"aws_msk_cluster\",\"resource_vendor\":\"AWS\"}"
+                }
+            ]
+        },
+    ]
+}
+```
+
+To enable bill-of-materials in the results use the `--bom` flag.
+
+**NOTE** Bill of Materials queries should always have:
+- `severity: "TRACE"`
+- `category: "Bill Of Materials"`
+- `issue_type: "BillOfMaterials"`
diff --git a/docs/commands.md b/docs/commands.md
@@ -42,53 +42,56 @@ Usage:
   kics scan [flags]
 
 Flags:
-      --cloud-provider strings       list of cloud providers to scan (aws, azure, gcp)
-      --config string                path to configuration file
-      --disable-full-descriptions    disable request for full descriptions and use default vulnerability descriptions
-      --exclude-categories strings   exclude categories by providing its name
-                                     cannot be provided with query inclusion flags
-                                     can be provided multiple times or as a comma separated string
-                                     example: 'Access control,Best practices'
-  -e, --exclude-paths strings        exclude paths from scan
-                                     supports glob and can be provided multiple times or as a quoted comma separated string
-                                     example: './shouldNotScan/*,somefile.txt'
-      --exclude-queries strings      exclude queries by providing the query ID
-                                     cannot be provided with query inclusion flags
-                                     can be provided multiple times or as a comma separated string
-                                     example: 'e69890e6-fce5-461d-98ad-cb98318dfc96,4728cd65-a20c-49da-8b31-9c08b423e4db'
-  -x, --exclude-results strings      exclude results by providing the similarity ID of a result
-                                     can be provided multiple times or as a comma separated string
-      --exclude-severities strings   exclude results by providing the severity of a result
-                                     can be provided multiple times or as a comma separated string
-                                     example: 'info,low'
-                                     example: 'fec62a97d569662093dbb9739360942f...,31263s5696620s93dbb973d9360942fc2a...'
-      --fail-on strings              which kind of results should return an exit code different from 0
-                                     accepts: high, medium, low and info
-                                     example: "high,low" (default [high,medium,low,info])
-  -h, --help                         help for scan
-      --ignore-on-exit string        defines which kind of non-zero exits code should be ignored
-                                     accepts: all, results, errors, none
-                                     example: if 'results' is set, only engine errors will make KICS exit code different from 0 (default "none")
-  -i, --include-queries strings      include queries by providing the query ID
-                                     cannot be provided with query exclusion flags
-                                     can be provided multiple times or as a comma separated string
-                                     example: 'e69890e6-fce5-461d-98ad-cb98318dfc96,4728cd65-a20c-49da-8b31-9c08b423e4db'
-      --input-data string            path to query input data files
-  -b, --libraries-path string        path to directory with libraries (default "./assets/libraries")
-      --minimal-ui                   simplified version of CLI output
-      --no-progress                  hides the progress bar
-      --output-name string           name used on report creations (default "results")
-  -o, --output-path string           directory path to store reports
-  -p, --path strings                 paths or directories to scan
-                                     example: "./somepath,somefile.txt"
-      --payload-lines                adds line information inside the payload when printing the payload file
-  -d, --payload-path string          path to store internal representation JSON file
-      --preview-lines int            number of lines to be display in CLI results (min: 1, max: 30) (default 3)
-  -q, --queries-path string          path to directory with queries (default "./assets/queries")
-      --report-formats strings       formats in which the results will be exported (all, glsast, html, json, pdf, sarif) (default [json])
-      --timeout int                  number of seconds the query has to execute before being canceled (default 60)
-  -t, --type strings                 case insensitive list of platform types to scan
-                                     (Ansible, AzureResourceManager, CloudFormation, Dockerfile, Kubernetes, OpenAPI, Terraform)
+  -m, --bom                           include bill of materials (BoM) in results output
+      --cloud-provider strings        list of cloud providers to scan (aws, azure, gcp)
+      --config string                 path to configuration file
+      --disable-full-descriptions     disable request for full descriptions and use default vulnerability descriptions
+      --disable-secrets               disable secrets scanning
+      --exclude-categories strings    exclude categories by providing its name
+                                      cannot be provided with query inclusion flags
+                                      can be provided multiple times or as a comma separated string
+                                      example: 'Access control,Best practices'
+  -e, --exclude-paths strings         exclude paths from scan
+                                      supports glob and can be provided multiple times or as a quoted comma separated string
+                                      example: './shouldNotScan/*,somefile.txt'
+      --exclude-queries strings       exclude queries by providing the query ID
+                                      cannot be provided with query inclusion flags
+                                      can be provided multiple times or as a comma separated string
+                                      example: 'e69890e6-fce5-461d-98ad-cb98318dfc96,4728cd65-a20c-49da-8b31-9c08b423e4db'
+  -x, --exclude-results strings       exclude results by providing the similarity ID of a result
+                                      can be provided multiple times or as a comma separated string
+                                      example: 'fec62a97d569662093dbb9739360942f...,31263s5696620s93dbb973d9360942fc2a...'
+      --exclude-severities strings    exclude results by providing the severity of a result
+                                      can be provided multiple times or as a comma separated string
+                                      example: 'info,low'
+      --fail-on strings               which kind of results should return an exit code different from 0
+                                      accepts: high, medium, low and info
+                                      example: "high,low" (default [high,medium,low,info])
+  -h, --help                          help for scan
+      --ignore-on-exit string         defines which kind of non-zero exits code should be ignored
+                                      accepts: all, results, errors, none
+                                      example: if 'results' is set, only engine errors will make KICS exit code different from 0 (default "none")
+  -i, --include-queries strings       include queries by providing the query ID
+                                      cannot be provided with query exclusion flags
+                                      can be provided multiple times or as a comma separated string
+                                      example: 'e69890e6-fce5-461d-98ad-cb98318dfc96,4728cd65-a20c-49da-8b31-9c08b423e4db'
+      --input-data string             path to query input data files
+  -b, --libraries-path string         path to directory with libraries (default "./assets/libraries")
+      --minimal-ui                    simplified version of CLI output
+      --no-progress                   hides the progress bar
+      --output-name string            name used on report creations (default "results")
+  -o, --output-path string            directory path to store reports
+  -p, --path strings                  paths or directories to scan
+                                      example: "./somepath,somefile.txt"
+      --payload-lines                 adds line information inside the payload when printing the payload file
+  -d, --payload-path string           path to store internal representation JSON file
+      --preview-lines int             number of lines to be display in CLI results (min: 1, max: 30) (default 3)
+  -q, --queries-path string           path to directory with queries (default "./assets/queries")
+      --report-formats strings        formats in which the results will be exported (all, glsast, html, json, pdf, sarif) (default [json])
+  -r, --secrets-regexes-path string   path to secrets regex rules configuration file
+      --timeout int                   number of seconds the query has to execute before being canceled (default 60)
+  -t, --type strings                  case insensitive list of platform types to scan
+                                      (Ansible, AzureResourceManager, CloudFormation, Dockerfile, Kubernetes, OpenAPI, Terraform)
 
 Global Flags:
       --ci                  display only log messages to CLI output (mutually exclusive with silent)

diff --git a/docs/dockerhub.md b/docs/dockerhub.md
@@ -51,6 +51,7 @@ Usage:
   kics scan [flags]
 
 Flags:
+  -m, --bom                           include bill of materials (BoM) in results output
       --cloud-provider strings        list of cloud providers to scan (aws, azure, gcp)
       --config string                 path to configuration file
       --disable-full-descriptions     disable request for full descriptions and use default vulnerability descriptions

diff --git a/mkdocs.yml b/mkdocs.yml
@@ -46,6 +46,7 @@ nav:
       - General Info: queries.md
       - Creating Queries: creating-queries.md
       - Passwords And Secrets: secrets.md
+      - Bill of Materials: bom.md
       - Queries List:
           - All: queries/all-queries.md
           - Ansible: queries/ansible-queries.md