Skip to content

[CCR] Fetches/indexing/writes/reads/bulks in status fields vs readers in task params #34703

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bleskes opened this issue Oct 22, 2018 · 4 comments
Labels
blocker :Distributed Indexing/CCR Issues around the Cross Cluster State Replication features v6.5.0

Comments

@bleskes
Copy link
Contributor

bleskes commented Oct 22, 2018

The status reports for the following tasks often refer to fetches (number_of_successful_fetches,number_of_failed_fetches, total_fetch_time_millis,fetch_exceptions). We also have a similar term (which is identical in meaning) call reads (number_of_concurrent_reads). We also have the generic operations_received and total_transferred_bytes and last_requested_seq_no.

On the write sides we have bulk (number_of_successful_bulk_operations, number_of_failed_bulk_operations) and writes (number_of_concurrent_writes, number_of_queued_writes <-- note the different semantics here).

I think we should align our terminologies across the various places. I personally like the read versus write and bulk vs ops. Howabout:

  • number_of_succesful_bulk_reads
  • number_of_failed_bulk_reads
  • total_bulk_read_time
  • number_of_ongoing_bulk_reads
  • total_operations_read
  • total_bytes_read
  • last_requested_seq_no_for_reads
  • number_of_successful_bulk_writes
  • number_of_failed_bulk_writes
  • number_of_ongoing_bulk_writes
  • number_of_queued_ops_to_write (not too happy with this one)

/cc @jasontedor @martijnvg

@bleskes bleskes added :Distributed Indexing/CCR Issues around the Cross Cluster State Replication features team-discuss labels Oct 22, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

@bleskes bleskes changed the title [CCR] Fetches/indexing/writes/reads/bulkd in status fields vs readers in task params [CCR] Fetches/indexing/writes/reads/bulks in status fields vs readers in task params Oct 23, 2018
@bleskes
Copy link
Contributor Author

bleskes commented Oct 24, 2018

@chrisronline FYI. What ever ends up being deciding here will need a quick update on the UI side.

@bleskes
Copy link
Contributor Author

bleskes commented Oct 24, 2018

@jasontedor @martijnvg and me iterated on this and came up with the following namings. I mark this as a blocker for 6.5.0:

Stats

remote_cluster
leader_index
follower_index
shard_id
leader_global_checkpoint
leader_max_seq_no
follower_global_checkpoint
follower_max_seq_no
last_requested_seq_no
outstanding_read_requests
outstanding_write_requests
write_buffer_size(_in_bytes)
write_buffer_operation_count
follower_mapping_version
total_read_time_millis
total_read_remote_exec_time_millis
successful_read_requests
failed_read_requests
operations_read
bytes_read
total_index_time_millis
successful_write_requests
failed_write_requests
operations_written
read_exceptions
time_since_last_read_millis

Params

remote_cluster
leader_index
max_read_request_operation_count
max_read_request_size
max_write_request_operation_count
max_write_request_size
max_outstanding_read_requests
max_outstanding_write_requests
max_write_buffer_size
max_write_buffer_count
max_retry_delay
read_poll_timeout

@chrisronline
Copy link
Contributor

chrisronline commented Oct 24, 2018

Looks good.

I'm going to list the fields we are using in the UI and my best guess as to the new field name:

  • fetch_exceptions -> read_exceptions
  • time_since_last_fetch_millis -> time_since_last_read_millis
  • number_of_operations_indexed -> operations_written
  • number_of_failed_fetches -> failed_read_requests

The rest seem unchanged.

Is this accurate on terms of the renames?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocker :Distributed Indexing/CCR Issues around the Cross Cluster State Replication features v6.5.0
Projects
None yet
Development

No branches or pull requests

4 participants