Skip to content

Pyiceberg Supports Concurrent Write Operation ? #1682

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
omkenge opened this issue Feb 19, 2025 · 2 comments
Open

Pyiceberg Supports Concurrent Write Operation ? #1682

omkenge opened this issue Feb 19, 2025 · 2 comments

Comments

@omkenge
Copy link
Contributor

omkenge commented Feb 19, 2025

Question

Fo ref link
https://iceberg.apache.org/docs/1.6.0/reliability/#concurrent-write-operations

@kevinjqliu
Copy link
Contributor

Yes, PyIceberg does support concurrent write. Iceberg uses Optimistic Concurrency Control (OCC) which requires failed writers to retry.
Some catalogs, such as rest, has built-in retries. Others might have no it out-of-the-box but you can retry the same way.

@Fokko
Copy link
Contributor

Fokko commented Feb 19, 2025

I've raised a ticket earlier this week to actually add this: #1678

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants