FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

bpo-36346: Add Py_DEPRECATED to deprecated unicode APIs by methane · Pull Request #20878 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

bpo-36346: Add Py_DEPRECATED to deprecated unicode APIs #20878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
methane merged 18 commits into from
Jun 17, 2020
Conversation 42 Commits 18 Checks 0 Files changed
Merged

bpo-36346: Add Py_DEPRECATED to deprecated unicode APIs #20878

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
b7bb169
bpo-36346: Uncomment Py_DEPRECATED for deprecated unicode APIs
methane Jun 15, 2020
4e88718
Deprecate _PyUnicode_AsUnicode
methane Jun 15, 2020
8f58264
Merge remote-tracking branch 'origin/master' into pep393-deprecation
methane Jun 16, 2020
6b9d12c
Supress deprecation warnings
methane Jun 16, 2020
10e1b13
Use inline functions. Undeprecate some APIs.
methane Jun 16, 2020
a703182
Revive commented deprecation
methane Jun 16, 2020
468a43b
supress one more warning
methane Jun 16, 2020
52802fe
Remove Py_UNICODE_MATCH and deprecate PyUnicode_WSTR_LENGTH
methane Jun 16, 2020
6e86236
Undeprecate APIs that doesn't use wstr
methane Jun 16, 2020
b9fa49a
Update _testcapimodule
methane Jun 16, 2020
30bed2d
Add news
methane Jun 17, 2020
4490cc7
Add what's new entry
methane Jun 17, 2020
8c2daae
fixup
methane Jun 17, 2020
cae4804
Update Misc/NEWS.d/next/C API/2020-06-17-11-24-00.bpo-36346.fTMr3S.rst
methane Jun 17, 2020
26cb789
Update Objects/unicodeobject.c
methane Jun 17, 2020
4390489
Update Misc/NEWS.d/next/C API/2020-06-17-11-24-00.bpo-36346.fTMr3S.rst
methane Jun 17, 2020
35d68db
fixup
methane Jun 17, 2020
b3ee751
avoid _impl suffix
methane Jun 17, 2020
File filter

Filter by extension

Filter by extension .c  (3) .h  (1) .rst  (2) All 3 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
11 changes: 11 additions & 0 deletions Doc/whatsnew/3.9.rst
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -1097,6 +1097,12 @@ Porting to Python 3.9
internal C API (``pycore_gc.h``).
(Contributed by Victor Stinner in :issue:`40241`.)

* The ``Py_UNICODE_COPY``, ``Py_UNICODE_FILL``, ``PyUnicode_WSTR_LENGTH``,
:c:func:`PyUnicode_FromUnicode`, :c:func:`PyUnicode_AsUnicode`,
``_PyUnicode_AsUnicode``, and :c:func:`PyUnicode_AsUnicodeAndSize` are
marked as deprecated in C. They have been deprecated by :pep:`393` since
Python 3.3.
(Contributed by Inada Naoki in :issue:`36346`.)

Removed
-------
Expand Down Expand Up @@ -1165,3 +1171,8 @@ Removed

* Remove ``_PyUnicode_ClearStaticStrings()`` function.
(Contributed by Victor Stinner in :issue:`39465`.)

* Remove ``Py_UNICODE_MATCH``. It has been deprecated by :pep:`393`, and
broken since Python 3.3. The :c:func:`PyUnicode_Tailmatch` function can be
used instead.
(Contributed by Inada Naoki in :issue:`36346`.)
46 changes: 23 additions & 23 deletions Include/cpython/unicodeobject.h
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -46,13 +46,17 @@
Py_UNICODE_ISDIGIT(ch) || \
Py_UNICODE_ISNUMERIC(ch))

#define Py_UNICODE_COPY(target, source, length) \
memcpy((target), (source), (length)*sizeof(Py_UNICODE))

#define Py_UNICODE_FILL(target, value, length) \
do {Py_ssize_t i_; Py_UNICODE *t_ = (target); Py_UNICODE v_ = (value);\
for (i_ = 0; i_ < (length); i_++) t_[i_] = v_;\
} while (0)
Py_DEPRECATED(3.3) static inline void
Py_UNICODE_COPY(Py_UNICODE *target, const Py_UNICODE *source, Py_ssize_t length) {
memcpy(target, source, length * sizeof(Py_UNICODE));
}

Py_DEPRECATED(3.3) static inline void
Py_UNICODE_FILL(Py_UNICODE *target, Py_UNICODE value, Py_ssize_t length) {
for (Py_ssize_t i = 0; i < length; i++) {
target[i] = value;
}
}

/* macros to work with surrogates */
#define Py_UNICODE_IS_SURROGATE(ch) (0xD800 <= (ch) && (ch) <= 0xDFFF)
Expand All @@ -67,14 +71,6 @@
/* low surrogate = bottom 10 bits added to DC00 */
#define Py_UNICODE_LOW_SURROGATE(ch) (0xDC00 + ((ch) & 0x3FF))

/* Check if substring matches at given offset. The offset must be
valid, and the substring must not be empty. */

#define Py_UNICODE_MATCH(string, offset, substring) \
Comment thread
methane marked this conversation as resolved.
((*((string)->wstr + (offset)) == *((substring)->wstr)) && \
((*((string)->wstr + (offset) + (substring)->wstr_length-1) == *((substring)->wstr + (substring)->wstr_length-1))) && \
!memcmp((string)->wstr + (offset), (substring)->wstr, (substring)->wstr_length*sizeof(Py_UNICODE)))

/* --- Unicode Type ------------------------------------------------------- */

/* ASCII-only strings created through PyUnicode_New use the PyASCIIObject
Expand Down Expand Up @@ -247,10 +243,6 @@ PyAPI_FUNC(int) _PyUnicode_CheckConsistency(
int check_content);

/* Fast access macros */
#define PyUnicode_WSTR_LENGTH(op) \
Comment thread
methane marked this conversation as resolved.
(PyUnicode_IS_COMPACT_ASCII(op) ? \
((PyASCIIObject*)op)->length : \
((PyCompactUnicodeObject*)op)->wstr_length)

/* Returns the deprecated Py_UNICODE representation's size in code units
(this includes surrogate pairs as 2 units).
Expand Down Expand Up @@ -445,6 +437,14 @@ enum PyUnicode_Kind {
(0xffffU) : \
(0x10ffffU)))))

Py_DEPRECATED(3.3)
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
return PyUnicode_IS_COMPACT_ASCII(op) ?
((PyASCIIObject*)op)->length :
((PyCompactUnicodeObject*)op)->wstr_length;
}
#define PyUnicode_WSTR_LENGTH(op) _PyUnicode_get_wstr_length((PyObject*)op)

vstinner Jun 17, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

You might use _PyObject_CAST() macro:

Suggested change
#define PyUnicode_WSTR_LENGTH(op) _PyUnicode_get_wstr_length((PyObject*)op)
#define PyUnicode_WSTR_LENGTH(op) _PyUnicode_get_wstr_length(_PyObject_CAST(op))


/* === Public API ========================================================= */

/* --- Plain Py_UNICODE --------------------------------------------------- */
Expand Down Expand Up @@ -543,7 +543,7 @@ PyAPI_FUNC(void) _PyUnicode_FastFill(
only allowed if u was set to NULL.

The buffer is copied into the new object. */
/* Py_DEPRECATED(3.3) */ PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
const Py_UNICODE *u, /* Unicode buffer */
Py_ssize_t size /* size of buffer */
);
Expand Down Expand Up @@ -572,13 +572,13 @@ PyAPI_FUNC(Py_UCS4) _PyUnicode_FindMaxChar (
Py_UNICODE buffer.
If the wchar_t/Py_UNICODE representation is not yet available, this
function will calculate it. */
/* Py_DEPRECATED(3.3) */ PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
PyObject *unicode /* Unicode object */
);

/* Similar to PyUnicode_AsUnicode(), but raises a ValueError if the string
contains null characters. */
PyAPI_FUNC(const Py_UNICODE *) _PyUnicode_AsUnicode(
Py_DEPRECATED(3.3) PyAPI_FUNC(const Py_UNICODE *) _PyUnicode_AsUnicode(
PyObject *unicode /* Unicode object */
);

Expand All @@ -587,7 +587,7 @@ PyAPI_FUNC(const Py_UNICODE *) _PyUnicode_AsUnicode(
If the wchar_t/Py_UNICODE representation is not yet available, this
function will calculate it. */

/* Py_DEPRECATED(3.3) */ PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicodeAndSize(
Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicodeAndSize(
PyObject *unicode, /* Unicode object */
Py_ssize_t *size /* location where to save the length */
);
Expand Down
4 changes: 4 additions & 0 deletions Misc/NEWS.d/next/C API/2020-06-17-11-24-00.bpo-36346.fTMr3S.rst
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Mark ``Py_UNICODE_COPY``, ``Py_UNICODE_FILL``, ``PyUnicode_WSTR_LENGTH``,
``PyUnicode_FromUnicode``, ``PyUnicode_AsUnicode``, ``_PyUnicode_AsUnicode``,
and ``PyUnicode_AsUnicodeAndSize`` as deprecated in C. Remove ``Py_UNICODE_MATCH``
which was deprecated and broken since Python 3.3.
10 changes: 10 additions & 0 deletions Modules/_testcapimodule.c
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -1668,6 +1668,10 @@ parse_tuple_and_keywords(PyObject *self, PyObject *args)

static volatile int x;

/* Ignore use of deprecated APIs */
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
Comment thread
methane marked this conversation as resolved.

/* Test the u and u# codes for PyArg_ParseTuple. May leak memory in case
of an error.
*/
Expand Down Expand Up @@ -1844,6 +1848,7 @@ test_widechar(PyObject *self, PyObject *Py_UNUSED(ignored))

Py_RETURN_NONE;
}
_Py_COMP_DIAG_POP

static PyObject *
unicode_aswidechar(PyObject *self, PyObject *args)
Expand Down Expand Up @@ -2064,6 +2069,10 @@ unicode_transformdecimaltoascii(PyObject *self, PyObject *args)
return PyUnicode_TransformDecimalToASCII(unicode, length);
}

/* Ignore use of deprecated APIs */
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS

static PyObject *
unicode_legacy_string(PyObject *self, PyObject *args)
{
Expand All @@ -2086,6 +2095,7 @@ unicode_legacy_string(PyObject *self, PyObject *args)

return u;
}
_Py_COMP_DIAG_POP

static PyObject *
getargs_w_star(PyObject *self, PyObject *args)
Expand Down
23 changes: 23 additions & 0 deletions Objects/unicodeobject.c
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,13 @@ extern "C" {
_PyUnicode_UTF8_LENGTH(op))
#define _PyUnicode_WSTR(op) \
(((PyASCIIObject*)(op))->wstr)

/* Don't use deprecated macro of unicodeobject.h */
#undef PyUnicode_WSTR_LENGTH
#define PyUnicode_WSTR_LENGTH(op) \
(PyUnicode_IS_COMPACT_ASCII(op) ? \
((PyASCIIObject*)op)->length : \
((PyCompactUnicodeObject*)op)->wstr_length)
#define _PyUnicode_WSTR_LENGTH(op) \
(((PyCompactUnicodeObject*)(op))->wstr_length)
#define _PyUnicode_LENGTH(op) \
Expand Down Expand Up @@ -970,11 +977,14 @@ ensure_unicode(PyObject *obj)
#include "stringlib/find_max_char.h"
#include "stringlib/undef.h"

_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
#include "stringlib/unicodedefs.h"
#include "stringlib/fastsearch.h"
#include "stringlib/count.h"
#include "stringlib/find.h"
#include "stringlib/undef.h"
_Py_COMP_DIAG_POP

/* --- Unicode Object ----------------------------------------------------- */

Expand Down Expand Up @@ -4097,6 +4107,11 @@ PyUnicode_AsUnicodeAndSize(PyObject *unicode, Py_ssize_t *size)
return w;
}

/* Deprecated APIs */

_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS

Py_UNICODE *
PyUnicode_AsUnicode(PyObject *unicode)
{
Expand Down Expand Up @@ -4135,6 +4150,8 @@ PyUnicode_GetSize(PyObject *unicode)
return -1;
}

_Py_COMP_DIAG_POP

Py_ssize_t
PyUnicode_GetLength(PyObject *unicode)
{
Expand Down Expand Up @@ -12364,6 +12381,8 @@ PyUnicode_IsIdentifier(PyObject *self)
return len && i == len;
}
else {
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
Py_ssize_t i = 0, len = PyUnicode_GET_SIZE(self);
if (len == 0) {
/* an empty string is not a valid identifier */
Expand Down Expand Up @@ -12401,6 +12420,7 @@ PyUnicode_IsIdentifier(PyObject *self)
}
}
return 1;
_Py_COMP_DIAG_POP
}
}

Expand Down Expand Up @@ -15955,7 +15975,10 @@ PyUnicode_AsUnicodeCopy(PyObject *unicode)
PyErr_BadArgument();
return NULL;
}
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
u = PyUnicode_AsUnicodeAndSize(unicode, &len);
_Py_COMP_DIAG_POP
if (u == NULL)
return NULL;
/* Ensure we won't overflow the size. */
Expand Down
4 changes: 4 additions & 0 deletions Python/getargs.c
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -1027,6 +1027,9 @@ convertsimple(PyObject *arg, const char **p_format, va_list *p_va, int flags,
case 'u': /* raw unicode buffer (Py_UNICODE *) */
case 'Z': /* raw unicode buffer or None */
{
// TODO: Raise DeprecationWarning
_Py_COMP_DIAG_PUSH
_Py_COMP_DIAG_IGNORE_DEPR_DECLS
Py_UNICODE **p = va_arg(*p_va, Py_UNICODE **);

if (*format == '#') {
Expand Down Expand Up @@ -1066,6 +1069,7 @@ convertsimple(PyObject *arg, const char **p_format, va_list *p_va, int flags,
arg, msgbuf, bufsize);
}
break;
_Py_COMP_DIAG_POP
}

case 'e': {/* encoded string */
Expand Down
Back | FazBrowse Home | New Git URL