[DOCS] Splits auditing.asciidoc into smaller files

Author: lcawl

x-pack/docs/en/security/auditing.asciidoc renamed to x-pack/docs/en/security/auditing/event-types.asciidoc

@@ -1,50 +1,10 @@
 [role="xpack"]
-[[auditing]]
-== Auditing security events
-
-You can enable auditing to keep track of security-related events such as
-authentication failures and refused connections. Logging these events enables you
-to monitor your cluster for suspicious activity and provides evidence in the
-event of an attack.
-
-[IMPORTANT]
-============================================================================
-Audit logs are **disabled** by default. To enable this functionality, you
-must set `xpack.security.audit.enabled` to `true` in `elasticsearch.yml`.
-============================================================================
-
-{Security} provides two ways to persist audit logs:
-
-* The <<audit-log-output, `logfile`>> output, which persists events to
-  a dedicated `<clustername>_access.log` file on the host's file system.
-* The <<audit-index, `index`>> output, which persists events to an Elasticsearch index.
-The audit index can reside on the same cluster, or a separate cluster.
-
-By default, only the `logfile` output is used when enabling auditing.
-To facilitate browsing and analyzing the events, you can also enable
-indexing by setting `xpack.security.audit.outputs` in `elasticsearch.yml`:
-
-[source,yaml]
-----------------------------
-xpack.security.audit.outputs: [ index, logfile ]
-----------------------------
-
-The `index` output type should be used in conjunction with the `logfile`
-output type Because it is possible for the `index` output type to lose
-messages if the target index is unavailable, the `access.log` should be
-used as the official record of events.
-
-NOTE: Audit events are batched for indexing so there is a lag before
-events appear in the index. You can control how frequently batches of
-events are pushed to the index by setting
-`xpack.security.audit.index.flush_interval` in `elasticsearch.yml`.
-
 [float]
 [[audit-event-types]]
 === Audit event types
 
-Each request may generate multiple audit events.
-The following is a list of the events that can be generated:
+When you are <<auditing,auditing security events>>, each request can generate 
+multiple audit events. The following is a list of the events that can be generated:
 
 |======
 | `anonymous_access_denied`         | | | Logged when a request is denied due to a missing
@@ -281,195 +241,3 @@ The log level determines  which attributes are included in a log entry.
 | `rule`              | The <<ip-filtering, IP filtering>> rule that denied
                         the request.
 |======
-
-[float]
-[[audit-log-output]]
-=== Logfile audit output
-
-The `logfile` audit output is the default output for auditing. It writes data to
-the `<clustername>_access.log` file in the logs directory.
-
-[float]
-[[audit-log-entry-format]]
-=== Log entry format
-
-The format of a log entry is:
-
-[source,txt]
-----------------------------------------------------------------------------
-[<timestamp>] [<local_node_info>] [<layer>] [<entry_type>] <attribute_list>
-----------------------------------------------------------------------------
-
-`<timestamp>`       ::      When the event occurred. You can configure the
-                            timestamp format in `log4j2.properties`.
-`<local_node_info>` ::      Information about the local node that generated
-                            the log entry. You can control what node information
-                            is included by configuring the
-                            {ref}/auditing-settings.html#node-audit-settings[local node info settings].
-`<layer>`           ::      The layer from which this event originated:
-                            `rest`, `transport` or `ip_filter`.
-`<entry_type>`      ::       The type of event that occurred: `anonymous_access_denied`,
-                            `authentication_failed`, `access_denied`, `access_granted`,
-                            `connection_granted`, `connection_denied`.
-`<attribute_list>`  ::      A comma-separated list of key-value pairs that contain
-                            data pertaining to the event. Formatted as
-                            `attr1=[val1], attr2=[val2]`. See <<audit-event-attributes,
-                            Audit Entry Attributes>> for the attributes that can be included
-                            for each type of event.
-
-[float]
-[[audit-log-settings]]
-=== Logfile output settings
-
-The events and some other information about what gets logged can be
-controlled using settings in the `elasticsearch.yml` file. See
-{ref}/auditing-settings.html#event-audit-settings[Audited Event Settings] and
-{ref}/auditing-settings.html#node-audit-settings[Local Node Info Settings].
-
-IMPORTANT: No filtering is performed when auditing, so sensitive data may be
-audited in plain text when including the request body in audit events.
-
-[[logging-file]]
-You can also configure how the logfile is written in the `log4j2.properties`
-file located in `CONFIG_DIR`. By default, audit information is appended to the
-`<clustername>_access.log` file located in the standard Elasticsearch `logs` directory
-(typically located at `$ES_HOME/logs`). The file rolls over on a daily basis.
-
-[float]
-[[audit-log-ignore-policy]]
-=== Logfile audit events ignore policies
-
-The comprehensive audit trail is necessary to ensure accountability. It offers tremendous
-value during incident response and can even be required for demonstrating compliance.
-
-The drawback of an audited system is represented by the inevitable performance penalty incurred.
-In all truth, the audit trail spends _I/O ops_ that are not available anymore for the user's queries.
-Sometimes the verbosity of the audit trail may become a problem that the event type restrictions,
-<<audit-log-settings, defined by `include` and `exclude`>>, will not alleviate.
-
-*Audit events ignore policies* are a finer way to tune the verbosity of the audit trail.
-These policies define rules that match audit events which will be _ignored_ (read as: not printed).
-Rules match on the values of attributes of audit events and complement the <<audit-log-settings, include/exclude>> method.
-Imagine the corpus of audit events and the policies chopping off unwanted events.
-
-IMPORTANT: When utilizing audit events ignore policies you are acknowledging potential
-accountability gaps that could render illegitimate actions undetectable.
-Please take time to review these policies whenever your system architecture changes.
-
-A policy is a named set of filter rules. Each filter rule applies to a single event attribute,
-one of the `users`, `realms`, `roles` or `indices` attributes. The filter rule defines
-a list of {ref}/query-dsl-regexp-query.html#regexp-syntax[Lucene regexp], *any* of which has to match the value of the audit
-event attribute for the rule to match.
-A policy matches an event if *all* the rules comprising it match the event.
-An audit event is ignored, therefore not printed, if it matches *any* policy. All other
-non-matching events are printed as usual.
-
-All policies are defined under the `xpack.security.audit.logfile.events.ignore_filters`
-settings namespace. For example, the following policy named _example1_ matches
-events from the _kibana_ or _admin_user_ principals **and** operating over indices of the
-wildcard form _app-logs*_:
-
-[source,yaml]
-----------------------------
-xpack.security.audit.logfile.events.ignore_filters:
-  example1:
-    users: ["kibana", "admin_user"]
-    indices: ["app-logs*"]
-----------------------------
-
-An audit event generated by the _kibana_ user and operating over multiple indices
-, some of which do not match the indices wildcard, will not match.
-As expected, operations generated by all other users (even operating only on indices that
-match the _indices_ filter) will not match this policy either.
-
-Audit events of different types may have <<audit-event-attributes, different attributes>>.
-If an event does not contain an attribute for which some policy defines filters, the
-event will not match the policy.
-For example, the following policy named _example2_, will never match `authentication_success` or
-`authentication_failed` events, irrespective of the user's roles, because these
-event schemas do not contain the `role` attribute:
-
-[source,yaml]
-----------------------------
-xpack.security.audit.logfile.events.ignore_filters:
-  example2:
-    roles: ["admin", "ops_admin_*"]
-----------------------------
-
-Likewise, any events of users with multiple roles, some of which do not match the
-regexps will not match this policy.
-
-For completeness, although practical use cases should be sparse, a filter can match
-a missing attribute of an event, using the empty string ("") or the empty list ([]).
-For example, the following policy will match events that do not have the `indices`
-attribute (`anonymous_access_denied`, `authentication_success` and other types) as well
-as events over the _next_ index.
-
-[source,yaml]
-----------------------------
-xpack.security.audit.logfile.events.ignore_filters:
-  example3:
-    indices: ["next", ""]
-----------------------------
-
-
-[float]
-[[audit-index]]
-=== Index audit output
-
-In addition to logging to a file, you can store audit logs in Elasticsearch
-rolling indices. These indices can be either on the same cluster, or on a
-remote cluster. You configure the following settings in
-`elasticsearch.yml` to control how audit entries are indexed. To enable
-this output, you need to configure the setting `xpack.security.audit.outputs`
-in the `elasticsearch.yml` file:
-
-[source,yaml]
-----------------------------
-xpack.security.audit.outputs: [ index, logfile ]
-----------------------------
-
-For more configuration options, see
-{ref}/auditing-settings.html#index-audit-settings[Audit log indexing configuration settings].
-
-IMPORTANT: No filtering is performed when auditing, so sensitive data may be
-audited in plain text when including the request body in audit events.
-
-[float]
-==== Audit index settings
-
-You can also configure settings for the indices that the events are stored in.
-These settings are configured in the `xpack.security.audit.index.settings` namespace
-in `elasticsearch.yml`. For example, the following configuration sets the
-number of shards and replicas to 1 for the audit indices:
-
-[source,yaml]
-----------------------------
-xpack.security.audit.index.settings:
-  index:
-    number_of_shards: 1
-    number_of_replicas: 1
-----------------------------
-
-[float]
-==== Forwarding audit logs to a remote cluster
-
-To index audit events to a remote Elasticsearch cluster, you configure
-the following `xpack.security.audit.index.client` settings:
-
-* `xpack.security.audit.index.client.hosts`
-* `xpack.security.audit.index.client.cluster.name`
-* `xpack.security.audit.index.client.xpack.security.user`
-
-For more information about these settings, see
-{ref}/auditing-settings.html#remote-audit-settings[Remote Audit Log Indexing Configuration Settings].
-
-You can pass additional settings to the remote client by specifying them in the
-`xpack.security.audit.index.client` namespace. For example, to allow the remote
-client to discover all of the nodes in the remote cluster you can specify the
-`client.transport.sniff` setting:
-
-[source,yaml]
-----------------------------
-xpack.security.audit.index.client.transport.sniff: true
-----------------------------

x-pack/docs/en/security/auditing/forwarding-logs.asciidoc

@@ -0,0 +1,24 @@
+[role="xpack"]
+[float]
+[[forwarding-audit-logfiles]]
+==== Forwarding audit logs to a remote cluster
+
+To index audit events to a remote Elasticsearch cluster, you configure
+the following `xpack.security.audit.index.client` settings:
+
+* `xpack.security.audit.index.client.hosts`
+* `xpack.security.audit.index.client.cluster.name`
+* `xpack.security.audit.index.client.xpack.security.user`
+
+For more information about these settings, see
+{ref}/auditing-settings.html#remote-audit-settings[Remote Audit Log Indexing Configuration Settings].
+
+You can pass additional settings to the remote client by specifying them in the
+`xpack.security.audit.index.client` namespace. For example, to allow the remote
+client to discover all of the nodes in the remote cluster you can specify the
+`client.transport.sniff` setting:
+
+[source,yaml]
+----------------------------
+xpack.security.audit.index.client.transport.sniff: true
+----------------------------

x-pack/docs/en/security/auditing/index.asciidoc

@@ -0,0 +1,15 @@
+
+:edit_url: https://github.com/elastic/elasticsearch/edit/{branch}/x-pack/docs/en/security/auditing/overview.asciidoc
+include::overview.asciidoc[]
+
+:edit_url: https://github.com/elastic/elasticsearch/edit/{branch}/x-pack/docs/en/security/auditing/event-types.asciidoc
+include::event-types.asciidoc[]
+
+:edit_url: https://github.com/elastic/elasticsearch/edit/{branch}/x-pack/docs/en/security/auditing/output-logfile.asciidoc
+include::output-logfile.asciidoc[]
+
+:edit_url: https://github.com/elastic/elasticsearch/edit/{branch}/x-pack/docs/en/security/auditing/output-index.asciidoc
+include::output-index.asciidoc[]
+
+:edit_url: https://github.com/elastic/elasticsearch/edit/{branch}/x-pack/docs/en/security/auditing/forwarding-logs.asciidoc
+include::forwarding-logs.asciidoc[]

x-pack/docs/en/security/auditing/output-index.asciidoc

@@ -0,0 +1,38 @@
+[role="xpack"]
+[float]
+[[audit-index]]
+=== Index audit output
+
+In addition to logging to a file, you can store audit logs in Elasticsearch
+rolling indices. These indices can be either on the same cluster, or on a
+remote cluster. You configure the following settings in
+`elasticsearch.yml` to control how audit entries are indexed. To enable
+this output, you need to configure the setting `xpack.security.audit.outputs`
+in the `elasticsearch.yml` file:
+
+[source,yaml]
+----------------------------
+xpack.security.audit.outputs: [ index, logfile ]
+----------------------------
+
+For more configuration options, see
+{ref}/auditing-settings.html#index-audit-settings[Audit log indexing configuration settings].
+
+IMPORTANT: No filtering is performed when auditing, so sensitive data may be
+audited in plain text when including the request body in audit events.
+
+[float]
+==== Audit index settings
+
+You can also configure settings for the indices that the events are stored in.
+These settings are configured in the `xpack.security.audit.index.settings` namespace
+in `elasticsearch.yml`. For example, the following configuration sets the
+number of shards and replicas to 1 for the audit indices:
+
+[source,yaml]
+----------------------------
+xpack.security.audit.index.settings:
+  index:
+    number_of_shards: 1
+    number_of_replicas: 1
+----------------------------