This doesn't work on Windows, as repr(value) escapes backslashes and uses single quotes, which TOML interprets as literal strings.

I think it would also be nice if the parser emitted dicts as tables:

[python.version_parts]
major = 3
minor = 11
micro = 5
releaselevel = 'final'
serial = 0

The following suggestion does both, but the fix for Windows is just changing repr(obj) to '{obj'}.

(This also involves changing Line 57 to toml_dump(f, config.items()).

This doesn't work on Windows, as repr(value) escapes backslashes and uses single quotes, which TOML interprets as literal strings.

Thank you for catching this! I actually wasn't aware of the difference between basic and literal strings in TOML. I don't think we should run into any such instances here, at least for now, given the small scope, but we should try to avoid that if possible.

I think it would also be nice if the parser emitted dicts as tables:

[python.version_parts]
major = 3
minor = 11
micro = 5
releaselevel = 'final'
serial = 0

Yeah, I don't have any very strong thoughts about it, so we can go with whatever makes the implementation simpler.

The following suggestion does both, but the fix for Windows is just changing repr(obj) to '{obj'}.

That causes other issues, like escaping quotes. I think I am just gonna make an escape table and call str.translate.

The problem is more that in the previous version, e.g. the executable string would be 'S:\\Development\\tmp\\venv\\Scripts\\python.exe' on Windows, which then was printed to the TOML file as executable = 'S:\\Development\\tmp\\venv\\Scripts\\python.exe', but then we hit the literal-vs-escaped string behaviour in TOML.

The PR as it stands now (with translation table) prints the following to TOML: executable = "S:\\\\Development\\\\tmp\\\\venv\\\\Scripts\\\\python.exe" -- the quotation marks are right but we've now got doubled-up backslashes!

I think that the fix is:

-    0x005C: r'\\\\',
+    0x005C: '\\\\',

A

This thread makes me want to vote more strongly for either JSON or simple key=value with no quoting or escaping.

TOML parsers are not universal, which means a lot of tools are going to end up hacking together just enough of a parser to make things work enough of the time. Better to go with either something that's universal-enough, or something that's trivial to parse.

I understand.

This thread makes me want to vote more strongly for either JSON or simple key=value with no quoting or escaping.

IMO the simple key=value option wouldn't be a great option for the following reasons:

• Being a custom format, it requires a custom parser anyway, which no matter how simple we intend it to be, it'd probably end up resulting in more user code
  • If we support different data types, the user implementation wouldn't be trivial, which I think goes against the purpose of having a "simple" mapping
  • If everything is a string, since we will still need to represent non-string data (eg. ints, bools, etc.), I think users would either end up implementing the type parsing code anyway, or it would be much more cumbersome for them to use the data, as they would likely still need to convert it to a useful type so that they can actually use the information
• The lack of sections makes organization worse, makes it less ergonomic to have direct maps to the sysconfig API 1, and makes it more cumbersome for users to use the data (eg. the version_parts field would need to be split in 5 different fields, on top of that, each likely with a large prefix)

That said, JSON is a reasonable choice, I just dislike how annoying it is to edit it, maybe even painful if you aren't very well familiarized with it. Not that is a big deal, it's just my personal opinion.

TOML parsers are not universal, which means a lot of tools are going to end up hacking together just enough of a parser to make things work enough of the time. Better to go with either something that's universal-enough, or something that's trivial to parse.

That is fair, I guess, though all major languages I am aware of have a TOML parser library, and looking at TOML's adoption rate I think that will only improve, so, realistically, I don't think this is really an issue. Though, to be clear, I do completely understand the concern.

Anyway, maybe I am just being a bit too opinionated here 🤣

Footnotes

1. I would like to have direct mappings, where possible, from this file to the data structures in the new sysconfig API. ↩

Taking this back to the issue #107956

Can we make these paths (optionally) relative from the directory containing the file? That way it can be static for most scenarios, instead of needing to be dynamically generated (and e.g. in the Windows Store package we can't dynamically generate it and also put it into the expected location, because it's read-only).

Alternatively, include a few variables like {prefix} that the reader can substitute themselves.

Yeah, I think that makes sense for use-cases like yours.

In some cases the standard library is split. Can we name a specific file it will be alongside? (e.g. the tests below expect it to be next to sysconfig.py)

Also, on Windows, this implies that everyone should hard-code Lib into their paths, because the only paths you can resolve without running Python are sys.prefix and sys.executable. (I'm not sure how best to go about locating the stdlib on other platforms either, to be honest, but I trust you for those.)

In some cases the standard library is split. Can we name a specific file it will be alongside? (e.g. the tests below expect it to be next to sysconfig.py)

Right, I will clarify it is the pure part of the standard library, which corresponds to the sysconfig stdlib path.

Also, on Windows, this implies that everyone should hard-code Lib into their paths, because the only paths you can resolve without running Python are sys.prefix and sys.executable. (I'm not sure how best to go about locating the stdlib on other platforms either, to be honest, but I trust you for those.)

I don't see a great solution for this, given that sys.prefix may contain multiple installations. My main two target use-cases are cross-compilation, where the user should be able to locate install-details.json themselves, and Python launchers, where, knowing that stdlib is within sys.prefix, should be able to search for the **/install-details.txt pattern in order to discover the available installation within a prefix.

The approach for Python launchers is not optimal, but it can be fixed later. Currently, I am just focusing on getting the file format down, and making the file available somewhere. Afterwards, we can look at the discoverability issue and consider for eg. having a global registry where these files could be additionally installed (eg. on Linux this could be the ~/.config/python/installations and /etc/python/installations paths), but that's a whole other issue, and I am trying to break things down into smaller chunks so that I can actually hope to make meaningful progress.

What do you think?

Will this script grow an option to reveal the location of the install-details.json file?

I think that's reasonable, but I'd rather put it in the sysconfig CLI, which is less problematic.

Except we need a way to locate the file without launching the runtime? Which is why python-config is a shell script and not a Python script.

You're right 🤦

I just remembered that now when looking at GH-77620.

For Windows, probably the best equivalent is to recommend a PEP 514 addition to specify the full path to install-details.json. Probably an InstallDetailsPath value alongside the existing ExecutablePath one (mentioned in this section).

This assumes that we're running in the target interpreter, yeah? So we need another way to generate this file for when we're cross-compiling (which is a normal part of the Windows build process).

Is there an easy way to have the script pick up all the makefile variables during a build without having to actually parse the makefile?

No, but we can add a CLI option to specify the value, like we do for executable. This way you just give the script the value when you call it in the Makefile, or other build system.

I'd like to make this script as standalone as possible.