Update cli documentation for search-alerts (#2051)

Mikaayenson · github-actions[bot] · commit 83f3f8f6e86a · 2022-06-24T14:01:43.000Z
* Add cli documentation for search-alerts and table fields (cherry picked from commit 4ef1a1a)
diff --git a/CLI.md b/CLI.md
@@ -1,16 +1,16 @@
 # Command Line Interface (CLI)
 
-This covers more advanced CLI use cases and workflows. To [get started](README.md#getting-started) with the CLI, reference 
-the [README](README.md). Basic use of the CLI such as [creating a rule](CONTRIBUTING.md#creating-a-rule-with-the-cli) or 
+This covers more advanced CLI use cases and workflows. To [get started](README.md#getting-started) with the CLI, reference
+the [README](README.md). Basic use of the CLI such as [creating a rule](CONTRIBUTING.md#creating-a-rule-with-the-cli) or
 [testing](CONTRIBUTING.md#testing-a-rule-with-the-cli) are referenced in the [contribution guide](CONTRIBUTING.md).
 
 
 ## Using a config file or environment variables
 
-CLI commands which are tied to Kibana and Elasticsearch are capable of parsing auth-related keyword args from a config 
-file or environment variables. 
+CLI commands which are tied to Kibana and Elasticsearch are capable of parsing auth-related keyword args from a config
+file or environment variables.
 
-If a value is set in multiple places, such as config file and environment variable, the order of precedence will be as 
+If a value is set in multiple places, such as config file and environment variable, the order of precedence will be as
 follows:
 * explicitly passed args (such as `--user joe`)
 * environment variables
@@ -35,10 +35,10 @@ EX: `DR_USER=joe`
 
 ## Importing rules into the repo
 
-You can import rules into the repo using the `create-rule` or `import-rules` commands. Both of these commands will 
-require that the rules are schema-compliant and able to pass full validation. The biggest benefit to using these 
-commands is that they will strip[*](#note) additional fields[**](#note-2) and prompt for missing required 
-fields. 
+You can import rules into the repo using the `create-rule` or `import-rules` commands. Both of these commands will
+require that the rules are schema-compliant and able to pass full validation. The biggest benefit to using these
+commands is that they will strip[*](#note) additional fields[**](#note-2) and prompt for missing required
+fields.
 
 Alternatively, you can manually place rule files in the directory and run tests to validate as well.
 
@@ -81,10 +81,10 @@ Options:
 ```
 
 The primary advantage of using this command is the ability to import multiple rules at once. Multiple rule paths can be
-specified explicitly with unlimited arguments, recursively within a directory using `-d/--directory`[*](#note-3), or 
+specified explicitly with unlimited arguments, recursively within a directory using `-d/--directory`[*](#note-3), or
 a combination of both.
 
-In addition to the formats mentioned using `create-rule`, this will also accept an `.ndjson`/`jsonl` file 
+In addition to the formats mentioned using `create-rule`, this will also accept an `.ndjson`/`jsonl` file
 containing multiple rules (as would be the case with a bulk export).
 
 This will also strip additional fields and prompt for missing required fields.
@@ -103,6 +103,7 @@ defined, otherwise arguments should be passed to the sub-command as:
 
 `python -m detection-rules kibana -u <username> -p <password> upload-rule <...>`
 
+
 ```console
 python -m detection_rules es -h
 
@@ -122,30 +123,111 @@ Commands:
   collect-events  Collect events from Elasticsearch.
 ```
 
+Providers are the name that Elastic Cloud uses to configure authentication in Kibana. When we create deployment, Elastic Cloud configures two providers by default: basic/cloud-basic and saml/cloud-saml (for SSO).
+
 ```console
 python -m detection_rules kibana -h
 
+█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
+█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
+█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
+
 Usage: detection_rules kibana [OPTIONS] COMMAND [ARGS]...
 
   Commands for integrating with Kibana.
 
 Options:
+  --ignore-ssl-errors TEXT
   --space TEXT                 Kibana space
-  -kp, --kibana-password TEXT
+  --provider-name TEXT         For cloud deployments, Elastic Cloud configures
+                               two providers by default: cloud-basic and
+                               cloud-saml (for SSO)
+  --provider-type TEXT         For cloud deployments, Elastic Cloud configures
+                               two providers by default: basic and saml (for
+                               SSO)
   -ku, --kibana-user TEXT
-  --cloud-id TEXT
-  -k, --kibana-url TEXT
+  --kibana-url TEXT
+  -kp, --kibana-password TEXT
+  -kc, --kibana-cookie TEXT    Cookie from an authed session
+  --cloud-id TEXT              ID of the cloud instance. Defaults the cloud
+                               provider to cloud-basic if this option is
+                               supplied
   -h, --help                   Show this message and exit.
 
 Commands:
-  upload-rule  Upload a list of rule .toml files to Kibana.
+  search-alerts  Search detection engine alerts with KQL.
+  upload-rule    Upload a list of rule .toml files to Kibana.
+```
+
+## Searching Kibana for Alerts
+
+Alerts stored in Kibana can be quickly be identified by searching with the `search-alerts` command.
+
+
+```console
+python -m detection_rules kibana search-alerts -h
+
+Kibana client:
+Options:
+  --ignore-ssl-errors TEXT
+  --space TEXT                 Kibana space
+  --provider-name TEXT         For cloud deployments, Elastic Cloud configures
+                               two providers by default: cloud-basic and
+                               cloud-saml (for SSO)
+  --provider-type TEXT         For cloud deployments, Elastic Cloud configures
+                               two providers by default: basic and saml (for
+                               SSO)
+  -ku, --kibana-user TEXT
+  --kibana-url TEXT
+  -kp, --kibana-password TEXT
+  -kc, --kibana-cookie TEXT    Cookie from an authed session
+  --cloud-id TEXT              ID of the cloud instance. Defaults the cloud
+                               provider to cloud-basic if this option is
+                               supplied
+
+Usage: detection_rules kibana search-alerts [OPTIONS] [QUERY]
+
+  Search detection engine alerts with KQL.
+
+Options:
+  -d, --date-range <TEXT TEXT>...
+                                  Date range to scope search
+  -c, --columns TEXT              Columns to display in table
+  -e, --extend                    If columns are specified, extend the
+                                  original columns
+  -h, --help                      Show this message and exit.
 ```
 
+Running the following command will print out a table showing any alerts that have been generated recently.
+`python3 -m detection_rules kibana --provider-name cloud-basic --kibana-url <url> --kibana-user <username> --kibana-password <password> search-alerts`
 
+```console
+
+█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
+█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
+█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█
+
+===================================================================================================================================
+ host                                 rule
+ hostname                             name                                                                @timestamp
+===================================================================================================================================
+ stryker-malwares-MacBook-Pro.local   Sudo Heap-Based Buffer Overflow Attempt                             2022-06-21T14:08:34.288Z
+ stryker-malwares-MacBook-Pro.local   Suspicious Automator Workflows Execution                            2022-06-21T13:58:30.857Z
+ stryker-malwares-MacBook-Pro.local   Privilege Escalation Enumeration via LinPEAS                        2022-06-21T13:33:18.218Z
+ stryker-malwares-MacBook-Pro.local   Privilege Escalation Enumeration via LinPEAS                        2022-06-21T13:28:14.685Z
+ stryker-malwares-MacBook-Pro.local   Potential Reverse Shell Activity via Terminal                       2022-06-21T12:53:00.234Z
+ stryker-malwares-MacBook-Pro.local   Potential Reverse Shell Activity via Terminal                       2022-06-21T12:53:00.237Z
+ stryker-malwares-MacBook-Pro.local   Potential Kerberos Attack via Bifrost                               2022-06-20T20:33:53.810Z
+ stryker-malwares-MacBook-Pro.local   Potential Kerberos Attack via Bifrost                               2022-06-20T20:33:53.813Z
+ stryker-malwares-MacBook-Pro.local   Potential Privilege Escalation via Root Crontab File Modification   2022-06-20T20:23:50.557Z
+ stryker-malwares-MacBook-Pro.local   Download and Execution of JavaScript Payload                        2022-06-20T20:18:46.211Z
+===================================================================================================================================
+
+```
 ## Uploading rules to Kibana
 
-Toml formatted rule files can be uploaded as custom rules using the `kibana upload-rule` command. To upload more than one 
-file, specify multiple files at a time as individual args. This command is meant to support uploading and testing of 
+Toml formatted rule files can be uploaded as custom rules using the `kibana upload-rule` command. To upload more than one
+file, specify multiple files at a time as individual args. This command is meant to support uploading and testing of
 rules and is not intended for production use in its current state.
 
 ```console
@@ -172,7 +254,7 @@ Options:
 (detection-rules-build) (base) ➜  detection-rules git:(rule-loader) ✗
 ```
 
-Alternatively, rules can be exported into a consolidated ndjson file which can be imported in the Kibana security app 
+Alternatively, rules can be exported into a consolidated ndjson file which can be imported in the Kibana security app
 directly.
 
 ```console
@@ -195,13 +277,13 @@ Options:
   -h, --help                      Show this message and exit.
 ```
 
-_*To load a custom rule, the proper index must be setup first. The simplest way to do this is to click 
+_*To load a custom rule, the proper index must be setup first. The simplest way to do this is to click
 the `Load prebuilt detection rules and timeline templates` button on the `detections` page in the Kibana security app._
 
 
 ## Converting between JSON and TOML
 
-[Importing rules](#importing-rules-into-the-repo) will convert from any supported format to toml. Additionally, the 
+[Importing rules](#importing-rules-into-the-repo) will convert from any supported format to toml. Additionally, the
 command `view-rule` will also allow you to view a converted rule without importing it by specifying the `--rule-format` flag.
 
 To view a rule in JSON format, you can also use the `view-rule` command with the `--api-format` flag, which is the default.
@@ -215,7 +297,7 @@ white space stripped, normalized, sorted, and indented, prior to their json conv
 table is also stripped out, as this is meant to be used only in the context of this repository and not in Kibana..
 
 Additionally, the `version` of the rule is added to the file prior to exporting it. This is done to restrict version bumps
-to occur intentionally right before we create a release. Versions are auto-incremented based on detected changes in 
+to occur intentionally right before we create a release. Versions are auto-incremented based on detected changes in
 rules. This is based on the hash of the rule in the following format:
 * sorted json
 * serialized
@@ -229,5 +311,5 @@ As a result, all cases where rules are shown or converted to JSON are not just s
 Most of the CLI errors will print a concise, user friendly error. To enable debug mode and see full error stacktraces,
 you can define `"debug": true` in your config file, or run `python -m detection-rules -d <commands...>`.
 
-Precedence goes to the flag over the config file, so if debug is enabled in your config and you run 
+Precedence goes to the flag over the config file, so if debug is enabled in your config and you run
 `python -m detection-rules --no-debug`, debugging will be disabled.
diff --git a/detection_rules/kbwrap.py b/detection_rules/kbwrap.py
@@ -96,7 +96,11 @@ def search_alerts(ctx, query, date_range, columns, extend):
     with kibana:
         alerts = [a['_source'] for a in Signal.search({'query': kql_query})['hits']['hits']]
 
-    table_columns = ['host.hostname', 'signal.rule.name', 'signal.status', 'signal.original_time']
+    table_columns = ['host.hostname', 'rule.name', '@timestamp']
+
+    # check for events with nested signal fields
+    if alerts and 'signal' in alerts[0]:
+        table_columns = ['host.hostname', 'signal.rule.name', 'signal.status', 'signal.original_time']
     if columns:
         columns = list(columns)
         table_columns = table_columns + columns if extend else columns
diff --git a/detection_rules/misc.py b/detection_rules/misc.py
@@ -338,14 +338,17 @@ def get_kibana_client(cloud_id, kibana_url, kibana_user, kibana_password, kibana
 
 client_options = {
     'kibana': {
-        'cloud_id': click.Option(['--cloud-id'], default=getdefault('cloud_id')),
+        'cloud_id': click.Option(['--cloud-id'], default=getdefault('cloud_id'),
+                                 help="ID of the cloud instance."),
         'kibana_cookie': click.Option(['--kibana-cookie', '-kc'], default=getdefault('kibana_cookie'),
                                       help='Cookie from an authed session'),
         'kibana_password': click.Option(['--kibana-password', '-kp'], default=getdefault('kibana_password')),
         'kibana_url': click.Option(['--kibana-url'], default=getdefault('kibana_url')),
         'kibana_user': click.Option(['--kibana-user', '-ku'], default=getdefault('kibana_user')),
-        'provider_type': click.Option(['--provider-type'], default=getdefault('provider_type')),
-        'provider_name': click.Option(['--provider-name'], default=getdefault('provider_name')),
+        'provider_type': click.Option(['--provider-type'], default=getdefault('provider_type'),
+                                      help="Elastic Cloud providers: basic and saml (for SSO)"),
+        'provider_name': click.Option(['--provider-name'], default=getdefault('provider_name'),
+                                      help="Elastic Cloud providers: cloud-basic and cloud-saml (for SSO)"),
         'space': click.Option(['--space'], default=None, help='Kibana space'),
         'ignore_ssl_errors': click.Option(['--ignore-ssl-errors'], default=getdefault('ignore_ssl_errors'))
     },
