Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADBDEV-6342: Refactor gprestore helper oidWithBatchList #105

Closed
wants to merge 27 commits into from

Conversation

RekGRpth
Copy link
Member

@RekGRpth RekGRpth commented Sep 23, 2024

Refactor gprestore helper oidWithBatchList

Currently, gprestore helper has one loop over the list of tables with already
defined batches. This creates difficulties for moving to the next table when a
skip file is detected. Also, such a loop is not easy to parallelize so that all
batches of the same table are processed sequentially by the same
process/thread. Therefore, this patch offers refactoring of the list and one
loop with division into two loops: an outer loop over tables and an inner loop
over batches. Now, one table is found only once in the list, and the second
argument is not the batch number, as it was before, but the number of batches
for the table (for replicated tables, one batch is always used). Also, the main
process pre-creates pipes only for the first batches. The logic for processing
skip files has also been significantly simplified.


It is easier to view the changes with the "Hide whitespace" option enabled.

@RekGRpth RekGRpth marked this pull request as ready for review September 26, 2024 05:02
Copy link

@Stolb27 Stolb27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NB! We don't merge this patch before SRS for dependent improvement will be accepted.

helper/helper.go Outdated Show resolved Hide resolved
utils/agent_remote.go Outdated Show resolved Hide resolved
restore/data.go Outdated Show resolved Hide resolved
helper/restore_helper.go Outdated Show resolved Hide resolved
helper/restore_helper.go Outdated Show resolved Hide resolved
helper/restore_helper.go Outdated Show resolved Hide resolved
whitehawk
whitehawk previously approved these changes Sep 30, 2024
@RekGRpth RekGRpth marked this pull request as draft October 1, 2024 08:14
@RekGRpth RekGRpth closed this Oct 1, 2024
@RekGRpth RekGRpth deleted the ADBDEV-6342 branch October 1, 2024 11:44
@RekGRpth RekGRpth restored the ADBDEV-6342 branch December 3, 2024 05:13
@RekGRpth RekGRpth reopened this Dec 3, 2024
@RekGRpth RekGRpth closed this Dec 6, 2024
@RekGRpth RekGRpth deleted the ADBDEV-6342 branch December 6, 2024 14:59
@RekGRpth RekGRpth restored the ADBDEV-6342 branch December 24, 2024 08:27
@RekGRpth RekGRpth reopened this Dec 24, 2024
@RekGRpth RekGRpth closed this Dec 25, 2024
@RekGRpth RekGRpth deleted the ADBDEV-6342 branch December 25, 2024 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants