Skip to content

Commit 2629a95

Browse files
authored
[DOCS] EQL: Document until keyword support (#59320) (#59408)
1 parent 85101fa commit 2629a95

File tree

3 files changed

+109
-5
lines changed

3 files changed

+109
-5
lines changed

docs/reference/eql/limitations.asciidoc

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,5 +41,3 @@ queries that contain:
4141
** {eql-ref}/pipes.html#sort[`sort`]
4242
** {eql-ref}/pipes.html#unique[`unique`]
4343
** {eql-ref}/pipes.html#unique-count[`unique_count`]
44-
45-
* The `until` {eql-ref}/sequences.html[sequence keyword]

docs/reference/eql/search.asciidoc

Lines changed: 25 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ PUT /sec_logs/_bulk?refresh
3232
{ "@timestamp": "2020-12-07T11:07:08.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "file", "id": "bYA7gPay", "sequence": 4 }, "file": { "accessed": "2020-12-07T11:07:08.000Z", "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe", "type": "file", "size": 16384 }, "process": { "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe" } }
3333
{"index":{"_index" : "sec_logs", "_id" : "5"}}
3434
{ "@timestamp": "2020-12-07T11:07:09.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "process", "id": "aR3NWVOs", "sequence": 5 }, "process": { "name": "regsvr32.exe", "path": "C:\\Windows\\System32\\regsvr32.exe" } }
35+
{"index":{"_index" : "sec_logs", "_id" : "6"}}
36+
{ "@timestamp": "2020-12-07T11:07:10.000Z", "agent": { "id": "8a4f500d" }, "event": { "category": "process", "id": "GTSmSqgz0U", "sequence": 6, "type": "termination" }, "process": { "name": "regsvr32.exe", "path": "C:\\Windows\\System32\\regsvr32.exe" } }
3537
----
3638
// TESTSETUP
3739
@@ -101,7 +103,7 @@ https://en.wikipedia.org/wiki/Unix_time[Unix epoch], in ascending order.
101103
"name": "cmd.exe",
102104
"path": "C:\\Windows\\System32\\cmd.exe"
103105
}
104-
},
106+
},
105107
"sort": [
106108
1607252645000
107109
]
@@ -396,6 +398,27 @@ contains the shared `agent.id` value for each matching event.
396398
}
397399
----
398400
// TESTRESPONSE[s/"took": 60/"took": $body.took/]
401+
402+
You can use the <<eql-until-keyword,`until` keyword>> to specify an expiration
403+
event for sequences. Matching sequences must end before this event.
404+
405+
The following request adds
406+
`until [ process where event.type == "termination" ]` to the previous EQL query.
407+
This ensures matching sequences end before a process termination event.
408+
409+
[source,console]
410+
----
411+
GET /sec_logs/_eql/search
412+
{
413+
"query": """
414+
sequence by agent.id with maxspan=1h
415+
[ file where file.name == "cmd.exe" ]
416+
[ process where stringContains(process.name, "regsvr32") ]
417+
until [ process where event.type == "termination" ]
418+
"""
419+
}
420+
----
421+
// TEST[s/search/search\?filter_path\=\-\*\.sequences\.\*events\.\*fields/]
399422
====
400423

401424
[discrete]
@@ -556,7 +579,7 @@ tiebreaker for events with the same timestamp.
556579
}
557580
----
558581
// TESTRESPONSE[s/"took": 34/"took": $body.took/]
559-
<1> The event's <<eql-search-api-timestamp-field,timestamp>>, converted to
582+
<1> The event's <<eql-search-api-timestamp-field,timestamp>>, converted to
560583
milliseconds since the https://en.wikipedia.org/wiki/Unix_time[Unix
561584
epoch]
562585
<2> The event's `event.id` value.

docs/reference/eql/syntax.asciidoc

Lines changed: 84 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -485,7 +485,7 @@ sequence by user.name
485485
----
486486
====
487487

488-
You can combine the `sequence by` and `with maxspan` keywords to constrain a
488+
You can combine the `sequence by` and `with maxspan` keywords to constrain a
489489
sequence by both field values and a timespan.
490490

491491
[source,eql]
@@ -513,6 +513,89 @@ sequence by user.name with maxspan=15m
513513
----
514514
====
515515

516+
[discrete]
517+
[[eql-until-keyword]]
518+
==== `until` keyword
519+
520+
You can use the `until` keyword to specify an expiration event for sequences.
521+
Matching sequences must end before this event, which is not included the
522+
results. If this event occurs within a sequence, the sequence is not considered
523+
a match.
524+
525+
[source,eql]
526+
----
527+
sequence
528+
[ event_category_1 where condition_1 ]
529+
[ event_category_2 where condition_2 ]
530+
...
531+
until [ event_category_2 where condition_2 ]
532+
----
533+
534+
.*Example*
535+
[%collapsible]
536+
====
537+
The following EQL sequence query uses the `until` keyword to end sequences
538+
before a process termination event. Process termination events have an event
539+
category of `process` and `event.type` value of `termination`.
540+
541+
[source,eql]
542+
----
543+
sequence
544+
[ file where file.extension == "exe" ]
545+
[ process where true ]
546+
until [ process where event.type == "termination" ]
547+
----
548+
====
549+
550+
[TIP]
551+
====
552+
The `until` keyword can be helpful when searching for process sequences in
553+
Windows event logs, such as those ingested using
554+
{winlogbeat-ref}/index.html[Winlogbeat].
555+
556+
In Windows, a process ID (PID) is unique only while a process is running. After
557+
a process terminates, its PID can be reused.
558+
559+
You can search for a sequence of events with the same PID value using the `by`
560+
and `sequence by` keywords.
561+
562+
.*Example*
563+
[%collapsible]
564+
=====
565+
The following EQL query uses the `sequence by` keyword to match a sequence of
566+
events that share the same `process.pid` value.
567+
568+
[source,eql]
569+
----
570+
sequence by process.pid
571+
[ process where process.name == "cmd.exe" ]
572+
[ process where process.name == "whoami.exe" ]
573+
----
574+
=====
575+
576+
However, due to PID reuse, this can result in a matching sequence that
577+
contains events across unrelated processes. To prevent false positives, you can
578+
use the `until` keyword to end matching sequences before a process termination
579+
event.
580+
581+
.*Example*
582+
[%collapsible]
583+
=====
584+
The following EQL query uses the `until` keyword to end sequences before
585+
`process` events with an `event.type` of `termination`. These events indicate a
586+
process has been terminated.
587+
588+
[source,eql]
589+
----
590+
sequence by process.pid
591+
[ process where process.name == "cmd.exe" ]
592+
[ process where process.name == "whoami.exe" ]
593+
until [ process where event.type == "termination" ]
594+
----
595+
=====
596+
597+
====
598+
516599
[discrete]
517600
[[eql-functions]]
518601
=== Functions

0 commit comments

Comments
 (0)