gh-145301: Fix double-free in hashlib and hmac module initialization by krylosov-aa · Pull Request #145321 · python/cpython · GitHub

Both hashlib and hmac modules have similar double-free bugs in their hashtable initialization code.

Problem

hashlib

In py_hashentry_table_new(), when _Py_hashtable_set() fails while adding an entry by its py_alias key (after successfully adding it by py_name), the code calls PyMem_Free(entry) before goto error. The error handler then calls _Py_hashtable_destroy() which frees the same entry again via py_hashentry_t_destroy_value().

hmac

In py_hmac_hinfo_ht_new(), the Py_HMAC_HINFO_LINK macro has the same issue. When py_hmac_hinfo_ht_add() fails for hashlib_name key (after successfully adding name), the code calls PyMem_Free(value) while the entry is already in the hashtable with refcnt=1.

Solution

hashlib

Remove the manual PyMem_Free(entry) call since the entry is already tracked by the hashtable under the py_name key and will be properly freed by _Py_hashtable_destroy().

hmac

Add a refcnt check before freeing: only call PyMem_Free(value) if value->refcnt == 0 (meaning it was never added to the table). If refcnt > 0, the entry is already in the table and will be freed by _Py_hashtable_destroy().

Tests

No test added. I couldn't find a practical way to trigger this bug since it requires _Py_hashtable_set() to fail during module initialization. This type of bug is best caught by sanitizers rather than unit tests.

The fix can be verified by running the reproducer provided by the issue author:

Build with ASan:

mkdir build-asan && cd build-asan
../configure --with-pydebug --with-address-sanitizer --without-pymalloc
make -j$(nproc)

Reproducer script (uaf_asan.py):

import subprocess
import sys

code = (
    "import sys, _testcapi\n"
    "if '_hashlib' in sys.modules:\n"
    "    del sys.modules['_hashlib']\n"
    "_testcapi.set_nomemory(40, 41)\n"
    "try:\n"
    "    import _hashlib\n"
    "except (MemoryError, ImportError):\n"
    "    pass\n"
    "finally:\n"
    "    _testcapi.remove_mem_hooks()\n"
)

result = subprocess.run(
    [sys.executable, '-c', code],
    capture_output=True, text=True, timeout=10
)

if result.returncode != 0:
    print(f"[*] CRASH confirmed (rc={result.returncode})")
    print(f"[*] {result.stderr.strip().split(chr(10))[-1]}")
else:
    print("[*] No crash (try different start values)")

Before fix:

$ ./build-asan/python uaf_asan.py
[*] CRASH confirmed (rc=-6)
[*] python: ../Include/internal/pycore_stackref.h:554: PyStackRef_FromPyObjectSteal: Assertion `obj != NULL' failed.

After fix: No crash.

Issue: _hashlib: Use-After-Free + Double Free in _hashopenssl.c py_hashentry_table_new() #145301

All commit authors signed the Contributor License Agreement.

Too verbose:

Suggested change

      
            Fix a double-free bug in :mod:`_hashlib` module initialization when
          
            ``_Py_hashtable_set()`` fails while adding an algorithm alias to the hash
          
            table after the primary name was already added
          
            :mod:`hashlib`: fix a crash when the C extension module initialization fails.

Please add the test repro as a regression test.

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Can you also check whether _hmac suffers from that? I don't remember whether I actually C/Ced this code or if I used another refcounting approach.

Can you also check whether _hmac suffers from that? I don't remember whether I actually C/Ced this code or if I used another refcounting approach.

I checked the HMAC implementation in _hashopenssl.c. It uses the same state->hashtable that is initialized by py_hashentry_table_new(). There's no separate hashtable for HMAC. So this fix covers both hash functions and HMAC.

I'm sorry, I meant the hmacmodule.c

I'm sorry, I meant the hmacmodule.c

Yes, hmacmodule.c has the same issue in py_hmac_hinfo_ht_new().

I can suggest the following solution:

#define Py_HMAC_HINFO_LINK(KEY)                                 \
        do {                                                    \
            int rc = py_hmac_hinfo_ht_add(table, KEY, value);   \
            if (rc < 0) {                                       \
+                if (value->refcnt == 0) {                      \
                    PyMem_Free(value);                          \
+                }                                              \
                goto error;                                     \
            }                                                   \
            else if (rc == 1) {                                 \
                value->refcnt++;                                \
            }                                                   \
        } while (0)
        Py_HMAC_HINFO_LINK(e->name);
        Py_HMAC_HINFO_LINK(e->hashlib_name);
#undef Py_HMAC_HINFO_LINK

Should I include this fix in the same PR or create a separate one?

Yes, please do so, and put a comment as well.

Please add the test repro as a regression test.

I investigated adding a regression test using _testcapi.set_nomemory, but found it's not feasible for this specific bug.

The set_nomemory() approach from the issue report triggers allocation failures globally, and the exact allocation number that hits our specific code path varies by platform/configuration.

Do you have any suggestions for how to reliably test this?

Suggested change

      
                            /* entry may already be in ht, will be freed by _Py_hashtable_destroy() */ \
          
                            /* entry may already be in ht, will be \ 
          
                             * freed by _Py_hashtable_destroy() */ \

correctly align the \. You can augment the column if it does not exceed 80 chars. If it does, properly wrap the comment. Tia.

Do you have any suggestions for how to reliably test this?

You have an ASASN reproducer, why not use it?

Put the imports alphabetically sorted.

Don't use this, use import_helper for that.

I don't think this is necessary.

Use assert_python_ok instead.

Can you create two NEWS entries, one for hashlib and for hmac please with the same sentence:

:mod:`<name>`: fix a crash when the initialization of the underlying C extension module fails.

Just put re-align the \ directly and put the comment on one line (you can remove the "will be freed by _Py_hashtable_destroy() and mention that it will be freed "upon exit").

sys is no more necessary

Don't do it at at the module level, it will skip the entire test module. Do it inside the test directly.

When is _testcapi lacking this function?

Do you have any suggestions for how to reliably test this?

You have an ASASN reproducer, why not use it?

I think that testing using set_nomemory(40, 41) is a bad idea because the number of memory allocations differs across platforms, and we cannot guarantee that allocations 40-41 will always occur in py_hashentry_table_new on all platforms.

I believe this is why the macOS Intel test crashes.

I don't know how to write a regression test for this scenario. I would suggest removing the test, since the fix is simple, comments have been left, and a regression seems unlikely to happen.

Yes, this seems flaky. It will change depending on the python's version and a platform.

nit: can we please revert this change if you decide to remove the test?

Oo for removig the test.

Thanks for making the requested changes!

@picnixz: please review the changes made to this pull request.

LGTM.

But I'm not sure that the NEWS entries are worth it since it's unlikely to get a memory allocation failure on importing these C extensions.

Thanks @krylosov-aa for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

…ation (pythonGH-145321) (cherry picked from commit 6acaf65) Co-authored-by: krylosov-aa <krylosov.andrew@gmail.com> pythongh-145301: Fix double-free in hashlib and hmac initialization

Sorry, @krylosov-aa and @gpshead, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 6acaf659ef0fdee131bc02f0b58685da039b5855 3.13

GH-145523 is a backport of this pull request to the 3.14 branch.

…zation (GH-145321) (#145523) gh-145301: Fix double-free in hashlib and hmac module initialization (GH-145321) (cherry picked from commit 6acaf65) gh-145301: Fix double-free in hashlib and hmac initialization Co-authored-by: krylosov-aa <krylosov.andrew@gmail.com>

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 FreeBSD14 3.14 (tier-3) has failed when building commit f426928.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1740/builds/1059) and take a look at the build logs.
Check if the failure is related to this commit (f426928) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1740/builds/1059

Failed tests:

test_httpservers

Failed subtests:

test_large_content_length_truncated - test.test_httpservers.CGIHTTPServerTestCase.test_large_content_length_truncated

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.14.opsec-fbsd14/build/Lib/test/test_httpservers.py", line 1139, in test_large_content_length_truncated
    res = self.request('/cgi-bin/file1.py', 'POST', b'x', headers)
  File "/home/buildbot/buildarea/3.14.opsec-fbsd14/build/Lib/test/test_httpservers.py", line 132, in request
    return self.connection.getresponse()
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.14.opsec-fbsd14/build/Lib/http/client.py", line 1450, in getresponse
    response.begin()
    ~~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.14.opsec-fbsd14/build/Lib/http/client.py", line 336, in begin
    version, status, reason = self._read_status()
                              ~~~~~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.14.opsec-fbsd14/build/Lib/http/client.py", line 305, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
                             " response")
http.client.RemoteDisconnected: Remote end closed connection without response

@krylosov-aa Could you please take care of manually backporting to 3.13?

@krylosov-aa Could you please take care of manually backporting to 3.13?

Yes, I will do it.

For 3.13, it is only hashopenssl that you need to change. HMAC HACL* did not exist yet.

This is a nice tricky fix :-) Thanks @krylosov-aa for the fix and thanks @raminfp for the bug report!

GH-145532 is a backport of this pull request to the 3.13 branch.

@picnixz @sergey-miryanov

I have created a manual backport for 3.13: see #145532

GH-145532 is a backport of this pull request to the 3.13 branch.

… (GH-145532) (cherry picked from commit 6acaf65)

…ation (pythonGH-145321) pythongh-145301: Fix double-free in hashlib and hmac initialization

krylosov-aa requested review from gpshead and picnixz as code owners February 27, 2026 16:10

bedevere-app Bot added the awaiting review label Feb 27, 2026

bedevere-app Bot mentioned this pull request Feb 27, 2026

_hashlib: Use-After-Free + Double Free in _hashopenssl.c py_hashentry_table_new() #145301

Closed

picnixz reviewed Feb 27, 2026

View reviewed changes

picnixz requested changes Feb 27, 2026

View reviewed changes

bedevere-app Bot added awaiting changes and removed awaiting review labels Feb 27, 2026

picnixz added needs backport to 3.13 only security fixes needs backport to 3.14 bugs and security fixes labels Feb 27, 2026

krylosov-aa changed the title gh-145301: Fix double-free in _hashlib module initialization gh-145301: Fix double-free in hashlib and hmac module initialization Feb 27, 2026

picnixz reviewed Feb 27, 2026

View reviewed changes

sobolevn reviewed Mar 2, 2026

View reviewed changes

krylosov-aa force-pushed the fix-issue-145301 branch from 154eda5 to c660a71 Compare March 2, 2026 08:15

picnixz mentioned this pull request Mar 2, 2026

gh-145376: Fix null pointer deref in md5module.c #145422

Merged

bedevere-app Bot added awaiting change review and removed awaiting changes labels Mar 2, 2026

bedevere-app Bot requested a review from picnixz March 2, 2026 21:23

picnixz approved these changes Mar 2, 2026

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting change review labels Mar 2, 2026

vstinner approved these changes Mar 4, 2026

View reviewed changes

gpshead approved these changes Mar 5, 2026

View reviewed changes

gpshead merged commit 6acaf65 into python:main Mar 5, 2026
52 of 55 checks passed

bedevere-app Bot removed the awaiting merge label Mar 5, 2026

miss-islington-app Bot assigned gpshead Mar 5, 2026

bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Mar 5, 2026

krylosov-aa mentioned this pull request Mar 5, 2026

[3.13] gh-145301: Fix double-free in hashlib initialization (GH-145321) #145532

Merged

bedevere-app Bot removed the needs backport to 3.13 only security fixes label Mar 5, 2026

encukou pushed a commit that referenced this pull request Mar 10, 2026

[3.13] gh-145301 : Fix double-free in hashlib initialization (GH-145321 )…

ba0d1b7

… (GH-145532) (cherry picked from commit 6acaf65)

ljfp pushed a commit to ljfp/cpython that referenced this pull request Apr 25, 2026

pythongh-145301 : Fix double-free in hashlib and hmac module initializ…

83fd6e1

…ation (pythonGH-145321) pythongh-145301: Fix double-free in hashlib and hmac initialization

               import tempfile
               import threading
               import unittest
+              import subprocess

+                      try:
+                          import _testcapi
+                          if not hasattr(_testcapi, 'set_nomemory'):
+                              self.skipTest('requires _testcapi.set_nomemory')
+                      except ImportError:
+                          self.skipTest('requires _testcapi')

+                      import_helper.import_module('_testcapi')
+                      code = textwrap.dedent("""
+                          import _testcapi
+                          _testcapi.set_nomemory(40, 41)

               from test.support import _4G, bigmemtest
               from test.support import hashlib_helper
-              from test.support.import_helper import import_fresh_module
+              from test.support import import_helper

	/* entry may already be in ht, will be freed by _Py_hashtable_destroy() */ \
	/* entry may already be in ht, will be \
	* freed by _Py_hashtable_destroy() */ \

		@@ -0,0 +1 @@
		Fix a crash when :mod:`hashlib` or :mod:`hmac` C extension module initialization fails.

		/* entry may already be in ht, will be freed by \
		_Py_hashtable_destroy() */ \


		requires_sha3 = unittest.skipUnless(_sha3, 'requires _sha3')

		_testcapi = import_helper.import_module('_testcapi')

                                       hash_type.value = False
                   @unittest.skipUnless(HASH is not None, 'need _hashlib')
+                  @unittest.skipUnless(hasattr(_testcapi, 'set_nomemory'),

		if '_hashlib' in sys.modules:
		del sys.modules['_hashlib']

gh-145301: Fix double-free in hashlib and hmac module initialization by krylosov-aa · Pull Request #145321 · python/cpython · GitHub

Repository navigation

gh-145301: Fix double-free in hashlib and hmac module initialization - #145321

gh-145301: Fix double-free in hashlib and hmac module initialization#145321gpshead merged 1 commit intopython:mainpython/cpython:mainfrom krylosov-aa:fix-issue-145301krylosov-aa/cpython:fix-issue-145301Copy head branch name to clipboard

Conversation

krylosov-aa commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

hashlib

hmac

Solution

hashlib

hmac

Tests

Uh oh!

python-cla-bot Bot commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

bedevere-app Bot commented Feb 27, 2026

Uh oh!

picnixz commented Feb 27, 2026

Uh oh!

krylosov-aa commented Feb 27, 2026

Uh oh!

picnixz commented Feb 27, 2026

Uh oh!

krylosov-aa commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

picnixz commented Feb 27, 2026

Uh oh!

krylosov-aa commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

picnixz commented Feb 27, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

krylosov-aa commented Feb 28, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

picnixz commented Mar 2, 2026

Uh oh!

bedevere-app Bot commented Mar 2, 2026

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app Bot commented Mar 5, 2026

Uh oh!

miss-islington-app Bot commented Mar 5, 2026

gh-145301: Fix double-free in hashlib and hmac module initialization#145321
gpshead merged 1 commit into
python:mainfrom
krylosov-aa:fix-issue-145301

krylosov-aa commented Feb 27, 2026 •
edited
Loading

python-cla-bot Bot commented Feb 27, 2026 •
edited
Loading

krylosov-aa commented Feb 27, 2026 •
edited
Loading

krylosov-aa commented Feb 27, 2026 •
edited
Loading