Skip to content

[DOCS] Defines data frame transform stats API objects #44197

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Feb 5, 2020
106 changes: 103 additions & 3 deletions docs/reference/transform/apis/get-transform-stats.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,109 @@ include::{docdir}/rest-api/common-parms.asciidoc[tag=size-transforms]
[[get-transform-stats-response]]
==== {api-response-body-title}

`transforms`::
(array) An array of statistics objects for {transforms}, which are
sorted by the `id` value in ascending order.
The API returns an array of statistics objects for {transforms}, which are
sorted by the `id` value in ascending order. All of these properties are
informational; you cannot update their values.

`checkpointing`::
(object) Contains statistics about <<transform-checkpoints,checkpoints>>.
`checkpointing`.`changes_last_detected_at`:::
(date) The timestamp when changes were last detected in the source indices.
`checkpointing`.`last`:::
(object) Contains statistics about the last completed checkpoint.
`checkpointing`.`last`.`checkpoint`::::
(integer) The sequence number for the checkpoint.
`checkpointing`.`last`.`time_upper_bound_millis`::::
(date) When using time-based synchronization, this timestamp indicates the
upper bound of data that is included in the checkpoint.
`checkpointing`.`last`.`timestamp_millis`::::
(date) The timestamp of the checkpoint, which indicates when the checkpoint
was created.
`checkpointing`.`next`:::
(object) Contains statistics about the next checkpoint that is currently in
progress. This object appears only when the {transform} `state` is `indexing`.
`checkpointing`.`next`.`checkpoint`::::
(integer) The sequence number for the checkpoint.
`checkpointing`.`next`.`checkpoint_progress`::::
(object) Contains statistics about the progress of the checkpoint. For example,
it lists the `total_docs`, `docs_remaining`, `percent_complete`,
`docs_processed`, and `docs_indexed`. This information is available only for
batch {transforms} and the first checkpoint of {ctransforms}.
`checkpointing`.`next`.`time_upper_bound_millis`::::
(date) When using time-based synchronization, this timestamp indicates the
upper bound of data that is included in the checkpoint.
`checkpointing`.`next`.`timestamp_millis`::::
(date) The timestamp of the checkpoint, which indicates when the checkpoint was
created.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checkpointing.next`:::
optional (object) Contains statistics about the next - currently in progress - checkpoint. This object only appears if the transform is currently processing data and only for the 1st checkpoint

It uses the same fields at last but has one more object:

checkpoint_progress::
(object) Contains statistics about the progress of the checkpoint.

Not sure how much we want to go into the details, the inner fields are:

  • total_docs
  • docs_remaining
  • percent_complete
  • docs_processed
  • docs_indexed

This information is only available for batch transforms and for the 1st checkpoint of a continuous transform.

`id`::
(string)
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id]

`node`::
(object) For started {transforms} only, the node upon which the {transform} is
started.
`node`.`attributes`:::
(object) A list of attributes for the node.
`node`.`ephemeral_id`:::
(string) The node ephemeral ID.
`node`.`id`:::
(string) The unique identifier of the node. For example, "0-o0tOoRTwKFZifatTWKNw".
`node`.`name`:::
(string) The node name. For example, `0-o0tOo`.
`node`.`transport_address`:::
(string) The host and port where transport HTTP connections are accepted. For
example, `127.0.0.1:9300`.
`state`::
(string) The status of the {transform}, which can be one of the following values:
+
--
* `aborting`: The {transform} is aborting.
* `failed`: The {transform} failed. For more information about the failure,
check the reason field.
* `indexing`: The {transform} is actively processing data and creating new
documents.
* `started`: The {transform} is running but not actively indexing data.
* `stopped`: The {transform} is stopped.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • aborting The {transform} is aborting.
  • stopping The {transform} is stopping.
  • failed The {transform} has failed. Check the reason field for further information.

* `stopping`: The {transform} is stopping.
--

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • reason::
    (string) The reason of the failure if the transform is in failed state.


`stats`::
(object) An object that provides statistical information about the {transform}.
`stats`.`documents_indexed`:::
(long) The number of documents that have been indexed into the destination index
for the {transform}.
`stats`.`documents_processed`:::
(long) The number of documents that have been processed from the source index of
the {transform}.
`stats`.`exponential_avg_checkpoint_duration_ms`:::
(double) Exponential moving average of the duration of the checkpoint, in milliseconds.
`stats`.`exponential_avg_documents_indexed`:::
(double) Exponential moving average of the number of new documents that have been
indexed.
`stats`.`exponential_avg_documents_processed`:::
(double) Exponential moving average of the number of documents that have been
processed.
`stats`.`index_failures`:::
(long) The number of indexing failures.
`stats`.`index_time_in_ms`:::
(long) The amount of time spent indexing, in milliseconds.
`stats`.`index_total`:::
(long) The number of indices created.
`stats`.`pages_processed`:::
(long) The number of search or bulk index operations processed. Documents are
processed in batches instead of individually.
`stats`.`search_failures`:::
(long) The number of search failures.
`stats`.`search_time_in_ms`:::
(long) The amount of time spent searching, in milliseconds.
`stats`.`search_total`:::
(long) The number of search operations on the source index for the {transform}.
`stats`.`trigger_count`:::
(long) The number of times the {transform} has been triggered by the scheduler.
For example, the scheduler triggers the {transform} indexer to check for updates
or ingest new data at an interval specified in the
<<put-transform-request-body,`frequency` property>>.

[[get-transform-stats-response-codes]]
==== {api-response-codes-title}
Expand Down