Skip to content

[PGPRO-5691] move mmapped ptrack map into shared postgres memory #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Feb 28, 2022

Conversation

kulaginm
Copy link
Member

@kulaginm kulaginm commented Feb 8, 2022

No description provided.

@kulaginm kulaginm requested a review from funny-falcon February 8, 2022 08:05
@kulaginm kulaginm marked this pull request as draft February 8, 2022 08:06
@kulaginm kulaginm marked this pull request as ready for review February 9, 2022 07:10
@kulaginm kulaginm requested a review from ololobus February 9, 2022 07:10
@kulaginm
Copy link
Member Author

kulaginm commented Feb 9, 2022

Вопрос ревьюирам: как думаете, нужно ли удалять строку
ok(! -f $node->data_dir . "/global/ptrack.map.mmap", "ptrack.map.mmap should be cleaned up");
из tap теста?

Comment on lines -522 to -530
/*
* XXX: for some reason assign_ptrack_map_size is called twice during the
* postmaster boot! First, it is always called with bootValue, so we use
* -1 as default value and no-op here. Next, it is called with the actual
* value from config. That way, we use 0 as an option for user to turn
* off ptrack and clean up all files.
*/
if (newval == -1)
return;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it not a problem anymore? I see you use 0 as a boot value and clean up files when ptrack_map_size == 0

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now do nothing (with files and memory) inside assign_ptrack_map_size() and use the value of ptrack_map_size after the re-assignment has passed.

Well, at least that's how I understand it.

@ololobus
Copy link
Contributor

ololobus commented Feb 9, 2022

ok(! -f $node->data_dir . "/global/ptrack.map.mmap", "ptrack.map.mmap should be cleaned up");

Yeah, you don't write this file, so I don't think that we need this check in tests anymore

* fix tap test (remove checking of ptrack.map.mmap file)
* cosmetic change (minimize diff with master)
engine.c Outdated
elog(ERROR, "ptrack read map: unexpected end of file while reading map file \"%s\", expected to read %zu, but read only %zu bytes",
ptrack_path, PtrackActualSize, readed);
}
else if (last_readed < 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Надо на EINTR проверить. И...

Посмотрел на искось код постгресса, практически везде просто делают continue. Т.е. ни какой специальной реакции типа CHECK_FOR_INTERRUPTS обычно нет.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Добавил проверку EINTR. CHECK_FOR_INTERRUPTS не добавлял.
Заодно убрал дублирующее readed += last_readed, которое каким-то чудом проскочило через предыдущие запуски тестов.

engine.c Outdated

#ifdef WIN32
/* Check ptrack version inside old ptrack map */
if (ptrack_map->version_num < PTRACK_COMPATIBLE_VERSION_NUM)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Здесь, думаю, всё-таки нужно != : вдруг следующая версия будет с багом, и кто-нибудь решит сдаунгрейдиться.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

исправил

engine.c Outdated
* Unconditionally update version
* This is usefull if we have read early compatible version of map
*/
ptrack_map->version_num = PTRACK_VERSION_NUM;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

А почему не PTRACK_COMPATIBLE_VERSION_NUM ?
Соответственно, это нужно только под if (is_new_map)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

исправил

if (ptrack_map_size != 0)
RequestAddinShmemSpace(PtrackActualSize);
else
ptrackCleanFiles();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

А к этому моменту уже был позван assign_ptrack_map_size ?

kulaginm added a commit to postgrespro/pg_probackup that referenced this pull request Feb 12, 2022
#471)

* [PGPRO-5691] ptrack-2.3: move mmapped ptrack map into shared postgres memory
In ptrack-2.3 ptrack.map.mmap will be removed and 'incorrect checksum' error
will not be fatal (postgrespro/ptrack#19)
* added test_corrupt_ptrack_map test compatibility with both version 2.2 and version 2.3 of ptrack
@codecov
Copy link

codecov bot commented Feb 12, 2022

Codecov Report

Merging #19 (8b30346) into master (2f15771) will increase coverage by 0.52%.
The diff coverage is 78.78%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #19      +/-   ##
==========================================
+ Coverage   88.55%   89.07%   +0.52%     
==========================================
  Files           2        2              
  Lines         428      421       -7     
==========================================
- Hits          379      375       -4     
+ Misses         49       46       -3     
Impacted Files Coverage Δ
engine.c 85.22% <75.00%> (+0.57%) ⬆️
ptrack.c 92.66% <88.88%> (-0.34%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2f15771...8b30346. Read the comment docs.

@kulaginm kulaginm merged commit a0d5a20 into master Feb 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants