bpo-45502: Fix test_shelve

[serhiy-storchaka]

Run test_shelve with all underlying dbm implementations and pickle protocols.
Also make test_shelve discoverable.

https://bugs.python.org/issue45502

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @ambv for commit a5436d9 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[ambv]

Gentoo buildbot failures unrelated per BPO-45436 (now fixed).

Re-running https://buildbot.python.org/all/#/builders/721/builds/50 to see if the segfault is reproducible. Looks related since it's a failure in test_shelve which is the subject of this PR.

[serhiy-storchaka]

Crashed again at same place (in _dbm.dbm.keys()). It is either a bug in the dbm.ndbm module or memory corruption in one of previous tests.

[serhiy-storchaka]

@pablogsal could you please run this test separately?

./python -m test -vuall test_shelve
./python -m test -vuall test_shelve -m TestProto0File_ndbmShelve
./python -m test -vuall test_shelve -m test.test_shelve.TestProto0File_ndbmShelve.test_read

[serhiy-storchaka]

I see that tests on this buildbot are ran with option -x test_dbm test_dbm_ndbm. Seems the dbm.ndbm module is broken on this buildbot.

[pablogsal]

I see that tests on this buildbot are ran with option -x test_dbm test_dbm_ndbm. Seems the dbm.ndbm module is broken on this buildbot.

This is https://bugs.python.org/issue45289 no? This should be deactivated on the buildbot:

https://github.com/python/buildmaster-config/blob/ec3ba6f4866ccc42d98137c82aba48632bfc03c4/master/custom/factories.py#L406

[serhiy-storchaka]

python/buildmaster-config#292

[ned-deily]

FWIW, I haven't looked much yet into these crashes but I haven't run into them myself yet. I suspect it's an interaction with the Homebrew-installed gdbm which macOS does not ship with. We also don't include gdbm in the python.org macOS binaries.

[ambv]

Since python/buildmaster-config#293 is landed, this can be merged as well.

[miss-islington]

Thanks @serhiy-storchaka for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9, 3.10.
🐍🍒⛏🤖

[miss-islington]

Sorry, @serhiy-storchaka and @ambv, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker b781cc3bfce7c052728b06aad9f1a467cced289d 3.10

[miss-islington]

Sorry @serhiy-storchaka and @ambv, I had trouble checking out the 3.9 backport branch.
Please backport using cherry_picker on command line.
cherry_picker b781cc3bfce7c052728b06aad9f1a467cced289d 3.9

[bedevere-bot]

GH-29305 is a backport of this pull request to the 3.10 branch.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Arch Linux Asan 3.9 has failed when building commit a043706.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/579/builds/244) and take a look at the build logs.
4. Check if the failure is related to this commit (a043706) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/579/builds/244

Summary of the results of the build (if available):

Click to see traceback logs

Traceback (most recent call last):
  File "/buildbot/buildarea/3.9.pablogsal-arch-x86_64.asan/build/Lib/multiprocessing/resource_tracker.py", line 201, in main
    cache[rtype].remove(name)
KeyError: '/psm_bbb15bda'