Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backup: add checkpointing to backup compactions #143849

Merged
merged 1 commit into from
Apr 8, 2025

Conversation

kev-cao
Copy link
Contributor

@kev-cao kev-cao commented Apr 2, 2025

This patch adds periodic checkpointing to backup compaction jobs.

Epic: None

Release note: None

@kev-cao kev-cao requested review from a team as code owners April 2, 2025 16:36
@kev-cao kev-cao requested review from jeffswenson and removed request for a team April 2, 2025 16:36
Copy link

blathers-crl bot commented Apr 2, 2025

Your pull request contains more than 1000 changes. It is strongly encouraged to split big PRs into smaller chunks.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@kev-cao kev-cao requested review from DrewKimball and removed request for a team April 2, 2025 16:36
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@kev-cao kev-cao force-pushed the backup/compaction-checkpointing branch 4 times, most recently from b6b21ae to ef63425 Compare April 4, 2025 18:19
@kev-cao kev-cao requested a review from msbutler April 4, 2025 18:20
Copy link
Collaborator

@msbutler msbutler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@kev-cao kev-cao force-pushed the backup/compaction-checkpointing branch 4 times, most recently from 96b9623 to 8300774 Compare April 7, 2025 16:42
@kev-cao kev-cao requested a review from msbutler April 7, 2025 17:23
@kev-cao kev-cao force-pushed the backup/compaction-checkpointing branch from 8300774 to ffd6abf Compare April 7, 2025 20:42
@kev-cao kev-cao force-pushed the backup/compaction-checkpointing branch from ffd6abf to 5601aef Compare April 7, 2025 23:38
This patch adds periodic checkpointing to backup compaction jobs.

Epic: None

Release note: None
@kev-cao kev-cao force-pushed the backup/compaction-checkpointing branch from 5601aef to c4f7fce Compare April 8, 2025 14:32
Copy link
Collaborator

@msbutler msbutler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! This was a hard test to write, but i think it turned out nicely.

@kev-cao
Copy link
Contributor Author

kev-cao commented Apr 8, 2025

TFTR!

bors r=msbutler

craig bot pushed a commit that referenced this pull request Apr 8, 2025
143849: backup: add checkpointing to backup compactions r=msbutler a=kev-cao

This patch adds periodic checkpointing to backup compaction jobs.

Epic: None

Release note: None

143985: backup: checkpoint after online restore link phase r=msbutler a=msbutler

This patch will prevent online restore from rerunning the link flow if it has already completed. This will be useful for restore with experimental copy, which runs the download phase in the same job as the link phase. At some point in the future, we could add finer grain checkpointing to the link phase, but that would require more work and thought.

Epic: none

Release note: none

144000: orchestration: released CockroachDB version 25.1.3. Next version: 25.1.4 r=celiala a=cockroach-teamcity


Release note: None
Epic: None
Release justification: non-production (release infra) change.


144037: sql/stats: bring back guard against non-zero NumRange in forecasts r=yuzefovich a=michae2

It occurred to me tonight that the code we removed in #143955 not only generated a sentry report, but also returned an error instead of producing a faulty histogram. I too hope that #93892 is now fixed, but it seems wise to at least keep some code that guards against faulty histograms, even if we don't think the sentry report is necessary any more.

Informs: #93892

Epic: None

Release note: None

144050: dr: alias streaming replication to PCR r=msbutler a=ajstorm

Add an alias in the original Streaming Replication Between Clusters RFC to PCR, for easier discovery.

I can see an argument for not merging this, as the original RFC might not bare resemblance to what was built in the end, but the RFC is basically undiscoverable today due to its antiquated naming.

Epic: none
Release note: None

144088: ci: prioritize `staging-*` builds higher r=jlinder a=rickystewart

These builds are high-priority backports and we want them to get through the queue as quickly as possible.

Epic: DEVINF-1449
Release note: None

Co-authored-by: Kevin Cao <[email protected]>
Co-authored-by: Michael Butler <[email protected]>
Co-authored-by: Justin Beaver <[email protected]>
Co-authored-by: Michael Erickson <[email protected]>
Co-authored-by: Adam Storm <[email protected]>
Co-authored-by: Ricky Stewart <[email protected]>
@craig
Copy link
Contributor

craig bot commented Apr 8, 2025

This PR was included in a batch that successfully built, but then failed to merge into master (it was a non-fast-forward update). It will be automatically retried.

@craig craig bot merged commit c285f81 into cockroachdb:master Apr 8, 2025
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants