Skip to content

Add CustomStatus Size Check for OOP model #3113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 27, 2025
Merged

Conversation

nytian
Copy link
Collaborator

@nytian nytian commented May 19, 2025

Resolves #2918

Azure Storage has limit on Table Property, which is 64KB. For string value and UTF16 encoded, it should be 32 KB or less. Larger value will cause error.

This PR adds a CustomStatus size check, so that when it's too large, fail the orchestration instead. The limit size is 16KB with UTF16 encoded to align with in-process model.

Pull request checklist

  • My changes do not require documentation changes
    • Otherwise: Documentation PR is ready to merge and referenced in pending_docs.md
  • My changes should not be added to the release notes for the next release
    • Otherwise: I've added my notes to release_notes.md
  • My changes do not need to be backported to a previous version
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • I have added all required tests (Unit tests, E2E tests)
  • My changes do not require any extra work to be leveraged by OutOfProc SDKs
    • Otherwise: That work is being tracked here: #issue_or_pr_in_each_sdk
  • My changes do not change the version of the WebJobs.Extensions.DurableTask package
    • Otherwise: major or minor version updates are reflected in /src/Worker.Extensions.DurableTask/AssemblyInfo.cs
  • My changes do not add EventIds to our EventSource logs
    • Otherwise: Ensure the EventIds are within the supported range in our existing Windows infrastructure. You may validate this with a deployed app's telemetry. You may also extend the range by completing a PR such as this one.
  • My changes should be added to v2.x branch.
    • Otherwise: This change applies exclusively to WebJobs.Extensions.DurableTask v3.x. It will be retained only in the dev and main branches and will not be merged into the v2.x branch.

@nytian nytian requested review from cgillum and bachuv May 19, 2025 21:30
@nytian nytian requested review from cgillum and bachuv May 22, 2025 20:04
// Azure Table Storage enforces a 32 KB limit if a property value is a UTF-16 encoded string.
// We apply a 16 KB limit here to align with our in-process model.
const int MaxCustomStatusSizeInKB = 16;
double customStatusSizeInKB = customStatus != null ? Encoding.Unicode.GetByteCount(customStatus) / 1024.0 : 0;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a big deal, but do we really need floating-point computation here? Consider defining the constant as 16*1024 and just comparing it with the GetByteCount result. This may be more performant (though the impact is probably negligible) but, more importantly, this will eliminate some mental overhead for the reader of this code.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review, Anatoli! This was changed like this because of this comment .

@nytian nytian merged commit 5bdc7e4 into dev May 27, 2025
16 checks passed
@nytian nytian deleted the nytian/custom-status-limit branch May 27, 2025 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Orchestrator stuck in running when saving large custom status values
4 participants