Skip to content

gh-94199: Remove hashlib.pbkdf2_hmac() Python implementation #94200

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 6 additions & 12 deletions Doc/library/hashlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -300,23 +300,17 @@ include a `salt <https://en.wikipedia.org/wiki/Salt_%28cryptography%29>`_.

>>> from hashlib import pbkdf2_hmac
>>> our_app_iters = 500_000 # Application specific, read above.
>>> dk = pbkdf2_hmac('sha256', b'password', b'bad salt'*2, our_app_iters)
>>> dk = pbkdf2_hmac('sha256', b'password', b'bad salt' * 2, our_app_iters)
>>> dk.hex()
'15530bba69924174860db778f2c6f8104d3aaf9d26241840c8c4a641c8d000a9'

.. versionadded:: 3.4

.. note::
Function only available when Python is compiled with OpenSSL.

A fast implementation of *pbkdf2_hmac* is available with OpenSSL. The
Python implementation uses an inline version of :mod:`hmac`. It is about
three times slower and doesn't release the GIL.

.. deprecated:: 3.10
.. versionadded:: 3.4

Slow Python implementation of *pbkdf2_hmac* is deprecated. In the
future the function will only be available when Python is compiled
with OpenSSL.
.. versionchanged:: 3.12
Function now only available when Python is built with OpenSSL. The slow
pure Python implementation has been removed.

.. function:: scrypt(password, *, salt, n, r, p, maxmem=0, dklen=64)

Expand Down
6 changes: 6 additions & 0 deletions Doc/whatsnew/3.12.rst
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,12 @@ Removed
use :func:`locale.format_string` instead.
(Contributed by Victor Stinner in :gh:`94226`.)

* :mod:`hashlib`: Remove the pure Python implementation of
:func:`hashlib.pbkdf2_hmac()`, deprecated in Python 3.10. Python 3.10 and
newer requires OpenSSL 1.1.1 (:pep:`644`): this OpenSSL version provides
a C implementation of :func:`~hashlib.pbkdf2_hmac()` which is faster.
(Contributed by Victor Stinner in :gh:`94199`.)


Porting to Python 3.12
======================
Expand Down
70 changes: 4 additions & 66 deletions Lib/hashlib.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@
algorithms_available = set(__always_supported)

__all__ = __always_supported + ('new', 'algorithms_guaranteed',
'algorithms_available', 'pbkdf2_hmac', 'file_digest')
'algorithms_available', 'file_digest')


__builtin_constructor_cache = {}
Expand Down Expand Up @@ -180,72 +180,10 @@ def __hash_new(name, data=b'', **kwargs):
try:
# OpenSSL's PKCS5_PBKDF2_HMAC requires OpenSSL 1.0+ with HMAC and SHA
from _hashlib import pbkdf2_hmac
__all__ += ('pbkdf2_hmac',)
except ImportError:
from warnings import warn as _warn
_trans_5C = bytes((x ^ 0x5C) for x in range(256))
_trans_36 = bytes((x ^ 0x36) for x in range(256))

def pbkdf2_hmac(hash_name, password, salt, iterations, dklen=None):
"""Password based key derivation function 2 (PKCS #5 v2.0)

This Python implementations based on the hmac module about as fast
as OpenSSL's PKCS5_PBKDF2_HMAC for short passwords and much faster
for long passwords.
"""
_warn(
"Python implementation of pbkdf2_hmac() is deprecated.",
category=DeprecationWarning,
stacklevel=2
)
if not isinstance(hash_name, str):
raise TypeError(hash_name)

if not isinstance(password, (bytes, bytearray)):
password = bytes(memoryview(password))
if not isinstance(salt, (bytes, bytearray)):
salt = bytes(memoryview(salt))

# Fast inline HMAC implementation
inner = new(hash_name)
outer = new(hash_name)
blocksize = getattr(inner, 'block_size', 64)
if len(password) > blocksize:
password = new(hash_name, password).digest()
password = password + b'\x00' * (blocksize - len(password))
inner.update(password.translate(_trans_36))
outer.update(password.translate(_trans_5C))

def prf(msg, inner=inner, outer=outer):
# PBKDF2_HMAC uses the password as key. We can re-use the same
# digest objects and just update copies to skip initialization.
icpy = inner.copy()
ocpy = outer.copy()
icpy.update(msg)
ocpy.update(icpy.digest())
return ocpy.digest()

if iterations < 1:
raise ValueError(iterations)
if dklen is None:
dklen = outer.digest_size
if dklen < 1:
raise ValueError(dklen)

dkey = b''
loop = 1
from_bytes = int.from_bytes
while len(dkey) < dklen:
prev = prf(salt + loop.to_bytes(4))
# endianness doesn't matter here as long to / from use the same
rkey = from_bytes(prev)
for i in range(iterations - 1):
prev = prf(prev)
# rkey = rkey ^ prev
rkey ^= from_bytes(prev)
loop += 1
dkey += rkey.to_bytes(inner.digest_size)

return dkey[:dklen]
pass


try:
# OpenSSL's scrypt requires OpenSSL 1.1+
Expand Down
10 changes: 1 addition & 9 deletions Lib/test/test_hashlib.py
Original file line number Diff line number Diff line change
Expand Up @@ -1096,15 +1096,7 @@ def _test_pbkdf2_hmac(self, pbkdf2, supported):
iterations=1, dklen=None)
self.assertEqual(out, self.pbkdf2_results['sha1'][0][0])

@unittest.skipIf(builtin_hashlib is None, "test requires builtin_hashlib")
def test_pbkdf2_hmac_py(self):
with warnings_helper.check_warnings():
self._test_pbkdf2_hmac(
builtin_hashlib.pbkdf2_hmac, builtin_hashes
)

@unittest.skipUnless(hasattr(openssl_hashlib, 'pbkdf2_hmac'),
' test requires OpenSSL > 1.0')
@unittest.skipIf(openssl_hashlib is None, "requires OpenSSL bindings")
def test_pbkdf2_hmac_c(self):
self._test_pbkdf2_hmac(openssl_hashlib.pbkdf2_hmac, openssl_md_meth_names)

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
:mod:`hashlib`: Remove the pure Python implementation of
:func:`hashlib.pbkdf2_hmac()`, deprecated in Python 3.10. Python 3.10 and
newer requires OpenSSL 1.1.1 (:pep:`644`): this OpenSSL version provides
a C implementation of :func:`~hashlib.pbkdf2_hmac()` which is faster. Patch
by Victor Stinner.