Skip to content

S3 Export: Ensure that default storage class is configured for multipart upload #12780

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zinal opened this issue Dec 19, 2024 · 1 comment

Comments

@zinal
Copy link
Member

zinal commented Dec 19, 2024

Some third-party S3 protocol implementations seem to require the storage class field to be specified for all uploads.
AWS S3 treats NOT_SET as STANDARD, however, internally AWS SDK for C++ just doesn't set corresponding header, while adds it to the SignedHeaders.
MinIO implementation treats it as error, returning to client error, for PutObject requests. For MinIO, this has been fixed in PR #1148

Recently other S3 implementations have been discovered having a similarly looking issue for CreateMultipartUpload requests.
In addition, #1148 seem to have been dropped from main by PR #10333. If the latter PR somehow fixes the issue, it needs to be ported to 24-3 and tested.

@pixcc
Copy link
Member

pixcc commented Dec 19, 2024

It might be cheap to test these S3 using the AWS CLI. You will need to send PutObject and CreateMultipartUpload without specifying --storage-class.

$ aws s3api put-object --key "put"

$ aws s3api create-multipart-upload --key "multipart"

If these requests succeed, there is a good chance that #10333 fixes this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants