gh-114099 - Add iOS testbed, plus Makefile target to invoke it. by freakboy3742 · Pull Request #115930 · python/cpython · GitHub

Part of the PEP 730 work to add iOS support.

This PR adds an iOS testbed project, plus the Makefile target to support running it from the command line. It is the last of the code that was originally submitted as #115063.

The bulk of the PR is an iOS Xcode project; open iOS/testbed/iOSTestbed.xcodeproj from the command line will open the project in Xcode. The project itself is a bare stub app with no UI, and an XCTest test suite that contains 1 test - "run the CPython test suite"; the XCTest passes or fails depending on the overall success or failure of the CPython test suite.

Most of the project files are as generated by the new project wizard; differences of note include:

iOS/testbed/iOSTestbed/main.m marks some signals as ignored that the CPython test suite exercises, but iOS doesn't like to be unhandled.
iOS/testbed/iOSTestbedTests/iOSTestbedTests.m is an Xcode test suite with a single test that initialises a CPython interpreter and runs the test module. Command line options need to be passed in by modifying the argv array at the start of the file, as there's no real analog for a test suite command line (at least, not that I've found)
Build scripts have been added to post-process the standard library into "framework" form.
The build setting for Enable Testability has been set to Yes; this prevents stripping of binaries in release configurations. This is required because an iOS app may not link against symbols, but it will use them.
The build setting for User Script Sandboxing has been set to No. This allows the post-processing scripts to modify and sign framework binaries.

There are 2 other changes:

An extra dependency was added to the frameworkinstallmobileheaders target; I discovered that parallel make install builds would fail because it would try to correct the header install before actually installing the headers.
The configure script has been modified to use the iOS wrapper binaries rather than gcc as a default compiler when CC et al aren't defined. This makes configure easier to invoke (as you just need to have the path correct), and makes the failure mode more predictable (as an attempt to compile with gcc will get quite far along before failing). Manually specifying overrides for CC et al remains available as an option.

Running the test suite

As of this PR, it is possible to build CPython as a framework, and start the test suite.

The new Makefile target is make testios. This target will run the test suite on an iPhone SE 3rd edition; this is a simulator that is reliably available, as it's a model that doesn't get updated all that often.

Running the testbed requires that an iOS framework build for a simulator target has been compiled and installed into the stub Python.xcframework folder that the testbed project contains. The minimum invocation for running the test suite is:

export PATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin" 
./configure --host=arm64-apple-ios12.0-simulator --build=arm64-apple-darwin --with-build-python=/path/to/python3.13 --enable-framework
make clean
make -j4 all
make testios

(adjusting as necessary to point to a appropriate build-python). A cold start of the iOS simulator takes 2-3 minutes, and for most of that time, the simulator appears to be doing nothing. Don't be alarmed when the console output for the test run says Testing started and then appears to do nothing for a long time. If you're running the test suite from inside Xcode, it's a lot faster once the simulator is warm.

However, if you invoke this target, the test suite will currently fail because it can't find binary modules at runtime. Correcting this will be the subject of the next PR.

Issue: Add support for iOS as a target platform #114099

Super excited for this feature 🎉
Thank you!

Suggested change

      
            .PHONY: testiOS
          
            testiOS:
          
            .PHONY: test-iOS
          
            test-iOS:

?

My thinking here was to be consistent with testuniversal, cleantest, buildbottest et al. However, I'm happy to change this name if there's a general preference for test-iOS over testiOS.

All existing makefile targets are lower-case, even acronyms like multissltest.

I guess that's true... although I suspect I'd twitch every time I need to spell it with capitalization that breaks the branding guide 😀

I've updated this to testios

I think this entire block is redundant, because this is already the default location that will be derived from config.home. In which case, the whole chain that passes the Python version from the configure script can be removed as well.

Agreed this block isn't needed. The testbed project still needs a version number, but I guess it's not critical that this version number be the same as the Python version, as it's not going to be a released app, and there are other details in the logs that identify the Python version, so I guess we can drop the configuration portion as well.

The curses and tkinter tests will be skipped automatically due to their stdlib modules not being built, so there's no need to exclude them here.

Overall, it looks pretty good to me. I like the adding of the default tool stub selections into configure. I might have more comments after the follow-on PRs to get a fully working import system and the test suite can run.

One thing that I would like you to give some thought to, if you haven't already, is to make it possible to run make testiOS when the build directory is not the source directory. As a perhaps unwritten goal, we try to support multiple builds with different configure args from one source directory with the ultimate goal of a read-only source directory. Requiring the writable testbed directory to be in the source directory hinders those goals. Even something as simple as having the make testiOS rule copy the iOS/testbed directory from the source directory to the build directory would be fine. If that presents formidable obstacles, we wouldn't necessarily need to solve it immediately but I'd like your opinion on it.

One thing that I would like you to give some thought to, if you haven't already, is to make it possible to run make testiOS when the build directory is not the source directory. As a perhaps unwritten goal, we try to support multiple builds with different configure args from one source directory with the ultimate goal of a read-only source directory.

That definitely makes sense as a goal; I think I've addressed that goal with my most recent commit.

To make this work, I've slightly modified the configure script. Previously, --enable-framework on iOS required the specification of an install location. In this version, a bare --enable-framework is now legal, and will create an "installed" version of the iOS framework in iOS/Frameworks/arm64-simulator (or similar). Specifying an explicit path will still work if you want.

Then, the testios target has install as a dependency; before running the test, it copies the testbed project into a timestamped copy in the ./build folder, and copies the "installed" framework from wherever it was constructed into the build version of the testbed. The project is then executed as before.

This means each testbed run is completely isolated from past runs; it also means there will be multiple copies of the testbed project and the full "installed" iOS framework - but most of them will be in the build folder, so they're easily cleaned up.

I've retained the instructions to purge the "source" version of the testbed as part of the clean target, in case anyone wants to use the older approach of "in tree" testbed builds - my feeling is that an in-tree build will be easier to manage for day-to-day development than the copied version in the build folder.

Does that approach makes sense? Is there an alternative approach that I haven't considered?

Thanks for addressing the previous comments. It's clear that, primarily due to the constraints of using an Xcode-provided simulator and of the necessary iOS framework structure, we are doing a bit of shoehorning here with a mixture of a Unix-style autoconf build and an Xcode project. But, if we can resolve the noted issues, I think this should be a reasonable compromise between the two styles.

I encountered this on Android as well. It turns out Python normally ignores these signals itself, but this is disabled by default with an isolated config. This can be fixed by adding the following line to the initialization code:

config.install_signal_handlers = 1;

Then it should be possible to remove these manual ignores.

…dir builds.

@ned-deily I've pushed an update that I think addresses all the issues you've flagged with the install dependency build location, and custom framework names; plus the signal configuration issue flagged by @mhsmith.

There's one additional piece - in #115774, @pablogsal added a new test module (_testexternalinspection.c) that was breaking the iOS build. AFAICT, iOS doesn't have a libproc.h, and the methods that are being appear to be Darwin kernel specific, but not exposed outside macOS. I've added extra pre-processing to make the test compile and pass.

There's one additional piece - in #115774, @pablogsal added a new test module (_testexternalinspection.c) that was breaking the iOS build. AFAICT, iOS doesn't have a libproc.h, and the methods that are being appear to be Darwin kernel specific, but not exposed outside macOS. I've added extra pre-processing to make the test compile and pass.

You can skip it in iOS if you want or add something to configure so it's not compiled in iOS

Thanks for resolving the comments nicely! It looks good to me now, with the caveat that, as advertised, the tests don't yet run under the simulator pending the additional changes to make extension module loading work. Unless someone has an objection, I think we can merge this PR and move on to the next.

Thanks for resolving the comments nicely! It looks good to me now, with the caveat that, as advertised, the tests don't yet run under the simulator pending the additional changes to make extension module loading work. Unless someone has an objection, I think we can merge this PR and move on to the next.

The good news is that the next PR in the chain adds the extension module loader, which means the test suite will run (although there are 4 failures for platform/sysconfig-related reasons). That PR is ready to go - I can push it as soon as this one lands. The PR after that one will resolve those 4 test failures.

The good news is that the next PR in the chain adds the extension module loader, which means the test suite will run (although there are 4 failures for platform/sysconfig-related reasons). That PR is ready to go - I can push it as soon as this one lands. The PR after that one will resolve those 4 test failures.

Hrm... after rebasing with main, I'm getting a weird error I wasn't getting a couple of days ago. Investigating...

Hrm... after rebasing with main, I'm getting a weird error I wasn't getting a couple of days ago. Investigating...

Is this the fatal_error_exit + 13 (pylifecycle.c:2817) that shows up in the iOSTestbed crash reports?

I did get that one - fixed it by rebuilding the host python. This one is a test failure the importlib loader tests that I wasn't seeing before...

Ok - found the problem. It was a weird import cache issue that I hadn't seen previously for some reason. PR incoming.

…pythongh-115930)

…oke it. (pythongh-115930)

…ke it. (pythongh-115930)

…oke it. (pythongh-115930)

…ke it. (pythongh-115930)

…oke it. (pythongh-115930)

…ke it. (pythongh-115930)

…oke it. (pythongh-115930)

Add iOS testbed, plus Makefile target to invoke it.

2ee1d5f

freakboy3742 requested review from corona10 and erlend-aasland as code owners February 26, 2024 06:18

bedevere-app Bot added the awaiting review label Feb 26, 2024

bedevere-app Bot mentioned this pull request Feb 26, 2024

Add support for iOS as a target platform #114099

Closed

freakboy3742 requested a review from ned-deily February 26, 2024 06:18

sobolevn reviewed Feb 26, 2024

View reviewed changes

Make better use of wildcards.

90066ae

mhsmith reviewed Feb 27, 2024

View reviewed changes

freakboy3742 added 2 commits February 28, 2024 14:04

Simplify testing executable.

392c006

Remove capitalization from the testios target.

cdf5df9

ned-deily reviewed Feb 28, 2024

View reviewed changes

Isolate testbed builds into the build directory.

6e768bd

Update README instructions for running the test suite.

0c720f2

ned-deily reviewed Mar 1, 2024

View reviewed changes

Comment thread Makefile.pre.in Outdated

Comment thread Makefile.pre.in Outdated

Comment thread configure.ac Outdated

Comment thread Makefile.pre.in

mhsmith reviewed Mar 5, 2024

View reviewed changes

freakboy3742 added 5 commits March 6, 2024 07:17

Merge branch 'main' into ios-testbed

5e63e8e

Modify makefile targets to account for renamed frameworks and non-src…

f946473

…dir builds.

Use config flag to enable signal handling.

ccb2f14

Disable testexternalinspection tests on iOS.

04afafb

Switch some clean targets to distclean.

be8f9b1

Move to a positive macOS check, rather than a negative iOS check.

24e2749

ned-deily approved these changes Mar 7, 2024

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting review labels Mar 7, 2024

ned-deily merged commit b33980a into python:main Mar 7, 2024

bedevere-app Bot removed the awaiting merge label Mar 7, 2024

freakboy3742 deleted the ios-testbed branch March 7, 2024 04:25

freakboy3742 mentioned this pull request Mar 7, 2024

gh-114099 - Add iOS framework loading machinery. #116454

Merged

adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 25, 2024

pythongh-114099 - Add iOS testbed, plus Makefile target to invoke it. (…

36ce85f

…pythongh-115930)

diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024

pythongh-114099 - Add iOS testbed, plus Makefile target to invoke it. (…

9d85c02

…pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Aug 5, 2024

[3.12] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

46bb44b

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Aug 5, 2024

[3.11] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

be30dbb

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Aug 5, 2024

[3.10] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

5a99a2a

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Aug 5, 2024

[3.9] pythongh-114099 - Add iOS testbed, plus Makefile target to invo…

54f0f8c

…ke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Sep 6, 2024

[3.12] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

4482258

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Sep 9, 2024

[3.12] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

25c9387

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Sep 9, 2024

[3.11] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

471ea34

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Sep 9, 2024

[3.10] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

cd690e8

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Sep 9, 2024

[3.9] pythongh-114099 - Add iOS testbed, plus Makefile target to invo…

fd5a4aa

…ke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Oct 9, 2024

[3.12] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

2908ca6

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Dec 13, 2024

[3.9] pythongh-114099 - Add iOS testbed, plus Makefile target to invo…

4a4b436

…ke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Dec 13, 2024

[3.10] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

7179330

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Dec 13, 2024

[3.12] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

7f51e73

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Dec 13, 2024

[3.11] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

99e7ae4

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Feb 3, 2026

[3.10] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

86de265

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Feb 3, 2026

[3.11] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

3ed8ce3

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Feb 3, 2026

[3.12] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

0ca95f5

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Jun 12, 2026

[3.11] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

11fb143

…oke it. (pythongh-115930)

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Jun 12, 2026

[3.12] pythongh-114099 - Add iOS testbed, plus Makefile target to inv…

1ca9871

…oke it. (pythongh-115930)

+                  // will be handled as if it were an argument to `python -m test`
+                  const char *argv[] = {
+                      "iOSTestbed", // argv[0] is the process that is running.
+                      "-uall,-gui,-curses",  // Enable most resources; GUI and curses tests won't work on iOS

+                  // Set the stdlib location for the Python interpreter
+                  path = [NSString stringWithFormat:@"%@/python/lib/python%@", resourcePath, py_version_string, nil];
+                  NSLog(@"Stdlib dir: %@", path);

+                      // iOS doesn't like uncaught signals.
+                      signal(SIGPIPE, SIG_IGN);
+                      signal(SIGXFSZ, SIG_IGN);

gh-114099 - Add iOS testbed, plus Makefile target to invoke it. by freakboy3742 · Pull Request #115930 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-114099 - Add iOS testbed, plus Makefile target to invoke it. - #115930

gh-114099 - Add iOS testbed, plus Makefile target to invoke it.#115930ned-deily merged 12 commits intopython:mainpython/cpython:mainfrom freakboy3742:ios-testbedfreakboy3742/cpython:ios-testbedCopy head branch name to clipboard

Conversation

freakboy3742 commented Feb 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Running the test suite

Uh oh!

sobolevn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ned-deily left a comment

Choose a reason for hiding this comment

Uh oh!

freakboy3742 commented Feb 29, 2024

Uh oh!

ned-deily left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

freakboy3742 commented Mar 6, 2024

Uh oh!

pablogsal commented Mar 6, 2024

Uh oh!

ned-deily left a comment

Choose a reason for hiding this comment

Uh oh!

freakboy3742 commented Mar 7, 2024

Uh oh!

freakboy3742 commented Mar 7, 2024

Uh oh!

ned-deily commented Mar 7, 2024

Uh oh!

freakboy3742 commented Mar 7, 2024

Uh oh!

freakboy3742 commented Mar 7, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gh-114099 - Add iOS testbed, plus Makefile target to invoke it.#115930
ned-deily merged 12 commits into
python:mainfrom
freakboy3742:ios-testbed

freakboy3742 commented Feb 26, 2024 •
edited
Loading