Docs: Improve guidance to developers on reflecting record failures

[94Sip]

What were you searching in the docs?

I was specifically looking for guidance in the docs on what the record_handler function should return to reflect a record (batch item) could not be handled successfully.

Is this related to an existing documentation section?

https://docs.powertools.aws.dev/lambda/python/latest/utilities/batch/#partial-failure-mechanics

How can we improve?

My recommendation is to specifically state that developers should raise an Exception (in python, for example) for any record that could not be processed successfully, and explicitly state the the "partial_process_response" object will handle any exceptions thrown by the record handler.

By looking at the "sample response" section, it implies that the "batchItemFailures" dictionary is returned, but it doesn't explicitly state how that response gets generated and what is required of the developer.

Got a suggestion in mind?

Responsibility of the developer/engineer implementing the record_handler method:

1. Allow Exceptions to be propagated to the partial_process_response object, and/or
2. Explicitly raise an Exception in the record handler

Either way, an Exception is used to notify the partial_process_response object that record handling was not successful.

From that point forward, the partial_process_response object will take care of formatting the "batchItemFailures" dictionary back to SQS.

Acknowledgment

• I understand the final update might be different from my proposed suggestion, or refused.

[heitorlessa]

Thank you so much for taking the time to report on Discord and create a beautifully detailed issue @94Sip - we'll ping once we have a draft for review.

Thanks!

[heitorlessa]

Took the liberty to improve other areas in the Batch documentation (diagrams!!), so I'll send the complete PR tomorrow.

Here's the initial draft for this specific section. I used code annotations (works on mobile too) to hide details of exception propagation besides what's written upfront. I'll also create a sequence diagram under Mechanics section to it can be referenced to more easily (like Idempotency).

🆕 Error handling section

Code annotation

Code annotation

---

🆕 Sequence diagrams section

SQS Standard queues

SQS FIFO

Kinesis and DynamoDB

---

Raw processing revamped plus samples

[heitorlessa]

PR is now up with all changes: #2857

[heitorlessa]

Closing manually as automation seems to have failed upon release: https://docs.powertools.aws.dev/lambda/python/latest/utilities/batch/#error-handling

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

[github-actions]

This is now released under 2.23.0 version!