FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

[3.6] bpo-31900: Fix localeconv() encoding for LC_NUMERIC (#4174) by vstinner · Pull Request #5192 · python/cpython · GitHub

Repository navigation

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension .c  (3) .h  (1) .rst  (4) All 3 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
10 changes: 10 additions & 0 deletions Doc/library/locale.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,16 @@ The :mod:`locale` module defines the following exception and functions:
| ``CHAR_MAX`` | Nothing is specified in this locale. |
+--------------+-----------------------------------------+

The function sets temporarily the ``LC_CTYPE`` locale to the ``LC_NUMERIC``
locale to decode ``decimal_point`` and ``thousands_sep`` byte strings if
they are non-ASCII or longer than 1 byte, and the ``LC_NUMERIC`` locale is
different than the ``LC_CTYPE`` locale. This temporary change affects other
threads.

.. versionchanged:: 3.6.5
The function now sets temporarily the ``LC_CTYPE`` locale to the
``LC_NUMERIC`` locale in some cases.


.. function:: nl_langinfo(option)

Expand Down
14 changes: 14 additions & 0 deletions Doc/library/stdtypes.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -1599,6 +1599,20 @@ expression support in the :mod:`re` module).
See :ref:`formatstrings` for a description of the various formatting options
that can be specified in format strings.

.. note::
When formatting a number (:class:`int`, :class:`float`, :class:`float`
and subclasses) with the ``n`` type (ex: ``'{:n}'.format(1234)``), the
function sets temporarily the ``LC_CTYPE`` locale to the ``LC_NUMERIC``
locale to decode ``decimal_point`` and ``thousands_sep`` fields of
:c:func:`localeconv` if they are non-ASCII or longer than 1 byte, and the
``LC_NUMERIC`` locale is different than the ``LC_CTYPE`` locale. This
temporary change affects other threads.

.. versionchanged:: 3.6.5
When formatting a number with the ``n`` type, the function sets
temporarily the ``LC_CTYPE`` locale to the ``LC_NUMERIC`` locale in some
cases.


.. method:: str.format_map(mapping)

Expand Down
8 changes: 8 additions & 0 deletions Doc/whatsnew/3.6.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -2346,3 +2346,11 @@ It has been replaced by the new ``make regen-all`` target.
(Contributed by Victor Stinner in :issue:`23404`.)

.. versionchanged:: 3.6.2


Notable changes in Python 3.6.5
===============================

The :func:`locale.localeconv` function now sets temporarily the ``LC_CTYPE``
locale to the ``LC_NUMERIC`` locale in some cases.
(Contributed by Victor Stinner in :issue:`31900`.)
5 changes: 5 additions & 0 deletions Include/fileutils.h
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,11 @@ PyAPI_FUNC(int) _Py_get_blocking(int fd);
PyAPI_FUNC(int) _Py_set_blocking(int fd, int blocking);
#endif /* !MS_WINDOWS */

PyAPI_FUNC(int) _Py_GetLocaleconvNumeric(
PyObject **decimal_point,
PyObject **thousands_sep,
const char **grouping);

#endif /* Py_LIMITED_API */

#ifdef __cplusplus
Expand Down
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
The :func:`locale.localeconv` function now sets temporarily the ``LC_CTYPE``
locale to the ``LC_NUMERIC`` locale to decode ``decimal_point`` and
``thousands_sep`` byte strings if they are non-ASCII or longer than 1 byte, and
the ``LC_NUMERIC`` locale is different than the ``LC_CTYPE`` locale. This
temporary change affects other threads.

Same change for the :meth:`str.format` method when formatting a number
(:class:`int`, :class:`float`, :class:`float` and subclasses) with the ``n``
type (ex: ``'{:n}'.format(1234)``).
32 changes: 26 additions & 6 deletions Modules/_localemodule.c
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -171,12 +171,6 @@ PyLocale_localeconv(PyObject* self)
RESULT(#i, x); \
} while (0)

/* Numeric information */
RESULT_STRING(decimal_point);
RESULT_STRING(thousands_sep);
x = copy_grouping(l->grouping);
RESULT("grouping", x);

/* Monetary information */
RESULT_STRING(int_curr_symbol);
RESULT_STRING(currency_symbol);
Expand All @@ -195,6 +189,32 @@ PyLocale_localeconv(PyObject* self)
RESULT_INT(n_sep_by_space);
RESULT_INT(p_sign_posn);
RESULT_INT(n_sign_posn);

/* Numeric information */
PyObject *decimal_point, *thousands_sep;
const char *grouping;
if (_Py_GetLocaleconvNumeric(&decimal_point,
&thousands_sep,
&grouping) < 0) {
goto failed;
}

if (PyDict_SetItemString(result, "decimal_point", decimal_point) < 0) {
Py_DECREF(decimal_point);
Py_DECREF(thousands_sep);
goto failed;
}
Py_DECREF(decimal_point);

if (PyDict_SetItemString(result, "thousands_sep", thousands_sep) < 0) {
Py_DECREF(thousands_sep);
goto failed;
}
Py_DECREF(thousands_sep);

x = copy_grouping(grouping);
RESULT("grouping", x);

return result;

failed:
Expand Down
77 changes: 77 additions & 0 deletions Python/fileutils.c
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -1597,3 +1597,80 @@ _Py_set_blocking(int fd, int blocking)
return -1;
}
#endif


int
_Py_GetLocaleconvNumeric(PyObject **decimal_point, PyObject **thousands_sep,
const char **grouping)
{
int res = -1;

struct lconv *lc = localeconv();

int change_locale = 0;
if (decimal_point != NULL &&
(strlen(lc->decimal_point) > 1 || ((unsigned char)lc->decimal_point[0]) > 127))
{
change_locale = 1;
}
if (thousands_sep != NULL &&
(strlen(lc->thousands_sep) > 1 || ((unsigned char)lc->thousands_sep[0]) > 127))
{
change_locale = 1;
}

/* Keep a copy of the LC_CTYPE locale */
char *oldloc = NULL, *loc = NULL;
if (change_locale) {
oldloc = setlocale(LC_CTYPE, NULL);
if (!oldloc) {
PyErr_SetString(PyExc_RuntimeWarning, "faild to get LC_CTYPE locale");
return -1;
}

oldloc = _PyMem_Strdup(oldloc);
if (!oldloc) {
PyErr_NoMemory();
return -1;
}

loc = setlocale(LC_NUMERIC, NULL);
if (loc != NULL && strcmp(loc, oldloc) == 0) {
loc = NULL;
}

if (loc != NULL) {
/* Only set the locale temporarilty the LC_CTYPE locale
if LC_NUMERIC locale is different than LC_CTYPE locale and
decimal_point and/or thousands_sep are non-ASCII or longer than
1 byte */
setlocale(LC_CTYPE, loc);
}
}

if (decimal_point != NULL) {
*decimal_point = PyUnicode_DecodeLocale(lc->decimal_point, NULL);
if (*decimal_point == NULL) {
goto error;
}
}
if (thousands_sep != NULL) {
*thousands_sep = PyUnicode_DecodeLocale(lc->thousands_sep, NULL);
if (*thousands_sep == NULL) {
goto error;
}
}

if (grouping != NULL) {
*grouping = lc->grouping;
}

res = 0;

error:
if (loc != NULL) {
setlocale(LC_CTYPE, oldloc);
}
PyMem_Free(oldloc);
return res;
}
15 changes: 4 additions & 11 deletions Python/formatter_unicode.c
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -707,18 +707,11 @@ get_locale_info(enum LocaleType type, LocaleInfo *locale_info)
{
switch (type) {
case LT_CURRENT_LOCALE: {
struct lconv *locale_data = localeconv();
locale_info->decimal_point = PyUnicode_DecodeLocale(
locale_data->decimal_point,
NULL);
if (locale_info->decimal_point == NULL)
if (_Py_GetLocaleconvNumeric(&locale_info->decimal_point,
&locale_info->thousands_sep,
&locale_info->grouping) < 0) {
return -1;
locale_info->thousands_sep = PyUnicode_DecodeLocale(
locale_data->thousands_sep,
NULL);
if (locale_info->thousands_sep == NULL)
return -1;
locale_info->grouping = locale_data->grouping;
}
break;
}
case LT_DEFAULT_LOCALE:
Expand Down

Back | FazBrowse Home | New Git URL