bpo-1294959 - better support for systems with /usr/lib64 by matejcik · Pull Request #3698 · python/cpython · GitHub

This introduces a new configure and sysconfig variable platsubdir, which is the name of subdirectory of prefix that holds platform-specific libraries. On many Linux distributions, there is separation between /usr/lib and /usr/lib64, as opposed to prefix vs exec_prefix, which is what Python is accustomed to.

This should improve the situation, and platsubdir will be used to differentiate library subdirectories below prefix.

Based on last comments in the bug report http://bugs.python.org/issue1294959 , I have renamed "platlibdir" to "platsubdir" -- IMHO it's more important to note that this is platform-specific than that it is a library directory. (otherwise we'd end up with platlibsubdir and that's horrible ;) )

https://bugs.python.org/issue1294959

didn't add a NEWS entry because I don't know how to do it ... is there a script that generates the file names for the NEWS.d structure? Or can I just drop a file called "bpo-1294959.txt" there?

@matejcik You will have to use blurb for the NEWS entry: https://pypi.python.org/pypi/blurb

In Fedora, we patch Python to support lib64 as follows:

https://src.fedoraproject.org/rpms/python3/blob/master/f/00102-lib64.patch (only applied on 64bit builds)
https://src.fedoraproject.org/rpms/python3/blob/master/f/00205-make-libpl-respect-lib64.patch (always applied)

This is only a note, I haven't looked at this PR more closely just yet, I'll try to compare the patches as soon as I can, to see if this PR would enable us to drop both of them.

Apparently, replacing those 2 patches with this one builds and tests fine. https://koji.fedoraproject.org/koji/taskinfo?taskID=22047740

news entry added

What can we do to make this happen?

I am a core dev now and I can commit to pushing this through.
Please rebase and re-check on Fedora; I can then do the sanity check on Ubuntu that Matthias requested on the issue.
If this ever again becomes stuck on the CPython side, ping me directly.

General comment: I wonder if it would make sense to make use of ax_set_default_paths_system.m4 as well

instead of --with-custom... just use --with. Also this is more commonly called --with-libdir

the whole point of the PR is that this is distinct from (how Python uses) libdir

okay, I've finally had a chance to re-read the original bug and this code. I'm not the biggest fan of introducing Yet Another variable and would prefer to fix how libdir gets used, but that seems like a bigger and more invasive change. Lets at least remove the custom portion of the name.

While I likely won't get to this for some time, it might be worth a more general conversation about what tunables we offer in configure, what they mean, and how they get used.

I don't like how we're special casing "lib" here. I'd prefer to always take the same code path and use some special token that gets replaced.

this is the same thing the code did before though.
i suppose we could introduce puresubdir as the other option and prefill it to lib somewhere, is that what you're suggesting?

hm, no, i misread which part of the code you're commenting. sorry.

This is not specific to Linux.

Unix then?

Unix-like?

@matejcik: I'm now interested to review your PR, but first I would like to see conflicts resolved. Would you like to fix conflicts? Or do you prefer that I create a new PR based on your PR?

This introduces a new configure and sysconfig variable `platsubdir`, which is the name of subdirectory of $prefix that holds platform-specific libraries. On many Linux distributions, there is separation between /usr/lib and /usr/lib64, which was causing some problems due to the way Python manages its install paths. Now, platsubdir is used to select the appropriate lib directory.

news entry news entry

@vstinner thanks, I fixed the basic conflicts.

Unfortunately, I see that the code (esp. in Modules/getpath.c) has drifted significantly, and this is no longer part of my day job. So I probably won't devote much time in getting this where it needs to go. Maybe @hroncok would be interested in taking over, or perhaps @mcepl ?

Not me. I can help @vstinner get this done and tested in Fedora, but that's it, sorry.

@matejcik @hroncok I hopefully will get to this later, but currently I struggle with fixing many packages failing on 3.7.*.

I can start working on it, should be having some more time soon.

Opened #11755 in order to move this forward.

Thank you to everyone for your work on this. I'm going to close this in favor of #11755 since that (at the time) was a rebase over master. If this one ends up being the better pull request to move the project forward, then this can be reopened.

PR #11755 is an updated version of this PR. (EDIT: oops, I missed @stratakis's comment.)

the-knights-who-say-ni added the CLA signed label Sep 22, 2017

bedevere-bot added the awaiting review label Sep 22, 2017

grimreaper suggested changes May 16, 2018

View reviewed changes

bedevere-bot added awaiting core review and removed awaiting review labels May 16, 2018

matejcik added 2 commits January 31, 2019 16:06

acknowledge myself

2a6e255

news entry news entry

matejcik force-pushed the build-64bit branch from f305dea to 2a6e255 Compare January 31, 2019 15:07

stratakis mentioned this pull request Feb 4, 2019

[WIP] bpo-1294959: Better support for systems with /usr/lib64 #11755

Closed

brettcannon added the type-feature A feature request or enhancement label Mar 29, 2019

csabella closed this May 23, 2019

vstinner mentioned this pull request Feb 11, 2020

bpo-1294959: Add --with-platlibdir to configure #18381

Merged

+              # platsubdir must be defined before LIBPL definition
+              AC_MSG_CHECKING(for custom platsubdir)
+              AC_ARG_WITH(custom-platsubdir,
+                  [AS_HELP_STRING([--with-custom-platsubdir=<libdirname>],

                               global_path = global_path.replace(sys.base_prefix, sys.prefix)
                               base = base.replace(sys.base_prefix, sys.prefix)
+                          if platsubdir != "lib":

		@@ -0,0 +1,3 @@
		On 64bit Linux systems that use ``$prefix/lib64``, this is now the default

bpo-1294959 - better support for systems with /usr/lib64 by matejcik · Pull Request #3698 · python/cpython · GitHub

Repository navigation

bpo-1294959 - better support for systems with /usr/lib64 - #3698

bpo-1294959 - better support for systems with /usr/lib64#3698matejcik wants to merge 2 commits intopython:masterpython/cpython:masterfrom matejcik:build-64bitmatejcik/cpython:build-64bitCopy head branch name to clipboard

Conversation

matejcik commented Sep 22, 2017 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matejcik commented Sep 22, 2017

Uh oh!

stratakis commented Sep 22, 2017

Uh oh!

hroncok commented Sep 23, 2017

Uh oh!

hroncok commented Sep 24, 2017

Uh oh!

matejcik commented Oct 11, 2017

Uh oh!

hroncok commented Nov 28, 2017

Uh oh!

encukou commented May 15, 2018

Uh oh!

grimreaper left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vstinner commented Jan 31, 2019

Uh oh!

matejcik commented Jan 31, 2019

Uh oh!

hroncok commented Jan 31, 2019

Uh oh!

mcepl commented Jan 31, 2019

Uh oh!

stratakis commented Jan 31, 2019

Uh oh!

stratakis commented Feb 4, 2019

Uh oh!

csabella commented May 23, 2019

Uh oh!

vstinner commented Feb 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

bpo-1294959 - better support for systems with /usr/lib64#3698
matejcik wants to merge 2 commits into
python:masterfrom
matejcik:build-64bit

matejcik commented Sep 22, 2017 •
edited by bedevere-bot
Loading

vstinner commented Feb 12, 2020 •
edited
Loading