Repository navigation
locale.setlocale() crashes on Windows for long locale name #137273
Description
Crash report
locale.setlocale(locale.LC_CTYPE, 'ks_IN.UTF-8@devanagari') crashes.
locale.setlocale(locale.LC_CTYPE, 'ks_IN.UTF8@devanagari') just raises a locale.Error.
It's not just about length. Standard locale names in Windows (like 'English_United States.1252') are pretty long.
Tested and reproduced in 3.12, 3.13, 3.14, and main, on Windows 10.
Linked PRs
Metadata
Metadata
Assignees
Labels
Projects
- Status
Activity
zooba commented on Aug 1, 2025
What's the actual crash? Is it asserting due to an invalid argument? Because we have the SUPPRESS_IPH macros to catch those.
If it's crashing inside the CRT and it's not a deliberate assertion, then we should also report it upstream and get it fixed there.
serhiy-storchaka commented on Aug 1, 2025
Yes, it is an assertion, but the SUPPRESS_IPH macros do not help. I already have a solution, in the process of writing the code.
pythongh-137273: Fix debug assertion failure in locale.setlocale() on…
pythongh-137273: Fix debug assertion failure in locale.setlocale() on…
malemburg commented on Aug 1, 2025
Does using "English_United States.1252" work with setlocale() ?
The limit of 16 chars you have in the PR seems rather small compared to typical lengths of local names.
Is this documented somewhere ?
@zooba Perhaps you could reach out to the folks at MS to ask why the function is crashing when passing more than 16 chars. This doesn't look right.
serhiy-storchaka commented on Aug 1, 2025
It happened when there were at least 16 characters after dot in the locale name. it was copied using wcscpy_s() to a fixed size array of 16 characters.
It is interesting that other parts of the locale name were also copied to fixed size arrays of length 64 and 85, but I did not get assertion failure when tried with long language, country or variant names.
serhiy-storchaka commented on Aug 1, 2025
Yes, it works. Even 'English.000000000001252' works. But 'English.0000000000001252' does not work, although it does not cause assertion failure.
16 is the number of elements in the szCodePage field of the __crt_locale_strings structure.
zooba commented on Aug 1, 2025
Yeah, it's obviously a deliberate crash due to overrunning the buffer (I've been reading the sources, not so easy to link all the parts though, sorry). No way to suppress it - it ends with a TerminateProcess call, so even if we could interrupt the error reporting, it'd still exit.
A bit of parsing to pre-check the length of the code page section (ends with -, _, . or \0, but not @) is probably the easiest way. Or maybe just strip off the @ since we know that's realistically the issue here.
Unfortunately the main parsing algorithm handles this properly, and the fallback one doesn't check the parsed length before attempting to copy. If the fallback could also fail safely due to a long code page, it'd be fine.
I can't find any known issues in our tracker at work about it, but there are a lot of crash reports, not just from this cause, but throughout the locale handling. I think we're right to be very defensive here (I wish we could just bypass it and not use the C runtime for this, but locale is CRT state and users will be relying on it being set there).
serhiy-storchaka commented on Aug 1, 2025
It happens only when parsing BCP-47 name, in parse_bcp47_code_page(). For other sections, language, region and script, there are explicit check for the length -- they can only be 2-3 or 4 characters. But there is no such check for code page (the part after dot). Other code is used for non-BCP-47 names, so it is not affected. But it has the same hard limit of 16 characters for code page. Even if this bug will be fixed, most likely the limit of 16 characters will remain, there will just be no assertion error. The test added in #137300 will still pass.
gh-137273: Fix debug assertion failure in locale.setlocale() on Windo…
pythongh-137273: Fix debug assertion failure in locale.setlocale() on…
pythongh-137273: Fix debug assertion failure in locale.setlocale() on…
[3.13] gh-137273: Fix debug assertion failure in locale.setlocale() o…
pythongh-137273: Fix debug assertion failure in locale.setlocale() on…
[3.14] gh-137273: Fix debug assertion failure in locale.setlocale() o…