I think it makes sense to ping pip folks about the test failure. @pfmoore @dstufft not something urgent, since 3.8 is still far away, but it would be great if pip didn't use the deprecated collections imports like collections.Mapping and switched to collections.abc.Mapping instead, thanks!.

This is actually in urllib3, we just bundle urllib3, it looks fixed in urllib3 and in pip and we just need to update CPython's bundled copy of pip.

we just need to update CPython's bundled copy of pip.

Great, thanks for looking into this!

#10598

The above has been merged, you should be able to rebase this PR.

This might cause make doctest to fail since some of the build dependencies might be incompatible https://bugs.python.org/issue35109

cc: @JulienPalard

Tests still are failing because of incompatibility in html5lib bundled with pip.

  File "<frozen zipimport>", line 259, in load_module
  File "/tmp/tmpcbwhlouc/pip-18.1-py2.py3-none-any.whl/pip/_vendor/html5lib/_trie/_base.py", line 3, in <module>
ImportError: cannot import name 'Mapping' from 'collections' (/home/travis/build/python/cpython/Lib/collections/__init__.py)

Note also syntax warnings produced by requests.

/tmp/tmpocpc284j/pip-18.1-py2.py3-none-any.whl/pip/_vendor/requests/status_codes.py:18: SyntaxWarning: invalid escape sequence \o

Warning in html5lib's _trie/_base.py seems to have been fixed upstream : html5lib/html5lib-python#403

Upstream issue I opened while testing : html5lib/html5lib-python#402

Requests also seem to be fixed : psf/requests#4728

So I guess pip's vendor folder needs to be updated for both libraries?

FWIW, it would be reasonable to delay this be one additional point release.

Also, if there are known major projects that still do the old imports, we should submit bug reports to those projects (I've already done some of these).

Also, if there are known major projects that still do the old imports,

Our docs build had this problem with a dependency that had this issue and now it's fixed upstream. I think it's better to do it now that gives more time and will help users and libraries who test their code with alpha and beta candidates if they have these imports.

@dstufft @pradyunsg https://bugs.python.org/issue35807 updates pip to 19.0 but I still see the vendored copy using html5lib that has this warning. Does pip update vendored version copy only after a release from html5lib? Should I file an issue in Pip's repo for this?

(sorry for the delay in responding; swamped in GitHub notifications currently)

Does pip update vendored version copy only after a release from html5lib?

Yep. pip downloads and vendors only versions released on PyPI.

Should I file an issue in Pip's repo for this?

This isn't necessary since we basically do a round of vendoring updates before a release which should cover this, but yeah, it won't hurt either. Please feel free to.

pip 19.0.3 still is not compatible with this PR.

Yea, there hasn't been an update for html5lib yet. :(

I pinged the author on #pypa-dev on Freenode today about this.

If we postpone the removal, can we change warning class from DeprecationWarning to ImportWarning to nosy more people?

Maybe, many Python developers doesn't care about DeprecationWarning after Python 3.2.

Ah. So that explains the documentation build failing. Thanks for the super quick response to a trivial question!

I gave it a try - the jinja issue is the last blocker. If I use an unreleased commit of jinja [1], tests are green [2].

[1] https://github.com/yan12125/cpython/commit/e6641707fb87774bf6359d8a584047cc94cc5b68#diff-c8dee9c3c068ad3447bbf459950a35b5
[2] https://travis-ci.org/yan12125/cpython/builds/567688247

The issue was fixed in 2.10.2 more than a year ago, but this version is not released yet.

That's surprising. Why was a tag created in Git with no public release on PyPI?

Jinja2 fix for collections.abc: pallets/jinja@593ee1e

Seems 2.10.2 is not tagged yet (or it was tagged but later removed): https://github.com/pallets/jinja/tags

There was an issue filed against Jinja2 in April asking when the fix would be included in a release, but the maintainers aren't saying much: pallets/jinja#973

A new version of Jinja is out - http://jinja.palletsprojects.com/en/2.10.x/changelog/#version-2-10-2. This pull request can go forward now.

And the git conflict seems trivial to fix :) c4106af

A regression is discovered in Jinja 2.10.2 (pallets/jinja#1069). It might be better to wait for 2.10.3.

The new (2.10.3) jinja release has been made! We're unblocked on this! :)

I gave it another try - with this PR rebased [1] and Jinja 2.10.3 installed, tests are still green [2].

[1] https://github.com/yan12125/cpython/commits/collections-abc (branch removed as this PR has been merged)
[2] https://travis-ci.org/yan12125/cpython/builds/593814893

Yay! We managed to do this in under an year! ^>^