FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat(runtime)!: overhaul native rendering and app architecture by owenthcarey · Pull Request #80 · pythonnative/pythonnative · GitHub

/ pythonnative Public

feat(runtime)!: overhaul native rendering and app architecture #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
owenthcarey merged 2 commits into from
Sep 5, 2026
Conversation 0 Commits 2 Checks 23 Files changed
Merged

feat(runtime)!: overhaul native rendering and app architecture #80

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ce90c2a
feat(runtime)!: overhaul native rendering and app architecture
owenthcarey Sep 5, 2026
64b3c84
fix(project,layout): fix simulator locks and portal layout
owenthcarey Sep 5, 2026
File filter

Filter by extension

Filter by extension .cpp  (22) .gradle  (2) .h  (65) .in  (1) .ini  (1) .java  (1) .js  (11) .json  (2) .kt  (69) .lock  (1) .map  (4) .md  (26) .modulemap  (2) .properties  (1) .py  (88) .swift  (58) .toml  (2) .ts  (4) .txt  (2) .xml  (1) .yaml  (10) .yml  (2) No extension  (2) dotfile  (2) All 24 file types selected
Only manifest files
Deleted files Viewed files
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
  •  
  •  
  •  
6 changes: 3 additions & 3 deletions .github/workflows/ci.yml
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ jobs:
# a named destination is missing, so resolve an available iPhone
# simulator by UDID instead of hardcoding a model name.
- name: Run PythonNativeKit tests
working-directory: src/pythonnative/templates/ios_template/PythonNativeKit
working-directory: src/pythonnative/native/ios
run: |
set -o pipefail
UDID="$(xcrun simctl list devices available -j | python3 -c '
Expand Down Expand Up @@ -95,5 +95,5 @@ jobs:
uses: gradle/actions/setup-gradle@v4

- name: Run pythonnative module tests
working-directory: src/pythonnative/templates/android_template
run: ./gradlew :pythonnative:testDebugUnitTest --no-daemon
working-directory: src/pythonnative/native/android
run: ../../templates/android_template/gradlew testDebugUnitTest --no-daemon
5 changes: 5 additions & 0 deletions .gitignore
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -165,3 +165,8 @@ cython_debug/

# macOS
.DS_Store
**/.cxx/
**/.build/
**/.gradle/
**/.kotlin/
**/.swiftpm/
73 changes: 61 additions & 12 deletions CONTRIBUTING.md
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -62,18 +62,23 @@ Unsolicited pull requests for issues that are already assigned or already have a
## Project layout (high‑level)

- `src/pythonnative/`: installable library and CLI
- `pythonnative/`: core cross‑platform UI components and utilities
- Core Python modules, UI components, and utilities
- `bridge/`: JSON codec and per-platform transports into the native rendering core
- `cli/`: `pn` command
- `project/`: config loading, template configuration, native plugin staging, and the builder behind `pn`
- `templates/`: Android/iOS project templates (bundled with the package)
- `ios_template/PythonNativeKit/`: Swift package with the iOS rendering core (component managers, gestures, animations, native modules) and its XCTest target
- `android_template/pythonnative/`: Gradle library module with the Android rendering core and its JUnit target
- `native/`: native libraries and their tests, bundled with the Python package
- `ios/`: `PythonNativeKit` Swift package and its XCTest target
- `android/`: Android rendering library and its JUnit target
- `yoga/`: vendored Yoga source, native build definitions, and the host Python binding
- `sdk/`: native contract definitions, code generation, and generator templates
- `devserver/static/`: browser preview assets, including the vendored Yoga JavaScript/WebAssembly distribution
- `templates/`: Android/iOS app shells; the builder stages native libraries into generated app projects
- `tests/`: unit tests for the library, plus the Maestro E2E suite
- `e2e/`: the comprehensive E2E suite (see [E2E tests](#e2e-tests-maestro) below and `tests/e2e/AGENTS.md`)
- `examples/`: runnable example apps
- `hello-world/`: minimal marketing demo
- `e2e-suite/`: comprehensive feature catalog that drives the Maestro E2E suite
- `inbox/`: offline reference app with a custom native extension
- `scripts/`: helper scripts (`check.sh`, `run-e2e.sh`, `check-e2e-coverage.py`)
- `README.md`, `pyproject.toml`: repo docs and packaging

Expand All @@ -83,7 +88,7 @@ Unsolicited pull requests for issues that are already assigned or already have a
- Prefer explicit, descriptive names; keep platform abstractions clean.
- Python never imports platform code. Everything that touches `UIView` or `android.view.View` lives in Swift (`PythonNativeKit`) or Kotlin (the `pythonnative` Gradle module) and is reached through `pythonnative.bridge`; see `docs/concepts/bridge.md` for the protocol. Bump `PROTOCOL_VERSION` on both sides when the wire format changes.
- Add/extend tests under `tests/` for new behavior. Native changes get XCTest / JUnit coverage next to the code they touch.
- Don't commit generated artifacts or large binaries; templates live under `src/pythonnative/templates/`.
- Commit source, reviewed generated contracts, dependency locks, and vendored dependencies as described in [Generated source and vendored dependencies](#generated-source-and-vendored-dependencies). Don't commit local build outputs or caches.
- Docstrings: Google style throughout. Ruff is configured with the Google
convention (`pydocstyle.convention = "google"`) and enforces the `D` rule
set on `src/pythonnative/`. See the
Expand All @@ -101,10 +106,53 @@ uv run black src examples tests # format
uv run --group docs mkdocs serve # preview the docs site locally

# native rendering core (macOS with Xcode for the Swift package; JDK 17 for Gradle)
(cd src/pythonnative/templates/ios_template/PythonNativeKit && xcodebuild test -scheme PythonNativeKit -destination 'platform=iOS Simulator,name=iPhone 15 Pro')
(cd src/pythonnative/templates/android_template && ./gradlew :pythonnative:testDebugUnitTest)
(cd src/pythonnative/native/ios && xcodebuild test -scheme PythonNativeKit -destination 'platform=iOS Simulator,name=iPhone 15 Pro')
(cd src/pythonnative/native/android && ../../templates/android_template/gradlew testDebugUnitTest)
```

## Generated source and vendored dependencies

Some generated files and third-party distribution files are intentional parts
of the source tree. Keep these in version control:

- Native contracts: `PNContracts`, `NativeProps`, and `NativeModules` in
`src/pythonnative/native/ios/Sources/PythonNativeKit/Generated/` and
`src/pythonnative/native/android/src/main/java/com/pythonnative/generated/`.
These support standalone native builds. Regenerate them from the Python
definitions and generator templates instead of editing them by hand.
- Example extension schemas, such as `examples/inbox/native/schema.json`.
Commit schema updates alongside their Python definitions and native
implementations.
- Dependency locks, including `uv.lock` and example apps' `pn.lock` files.
- Vendored Yoga source and browser distribution files under
`src/pythonnative/native/yoga/` and `src/pythonnative/devserver/static/yoga/`.
The browser distribution includes its JavaScript/WebAssembly runtime, type
declarations, and source maps. Preserve upstream licenses and document the
version, source, and any local modifications when updating vendored code.

After changing built-in contracts or their generators, regenerate the checked-in
native files from the repository root:

```bash
uv run pn codegen --output build/contracts
cp build/contracts/{PNContracts,NativeProps,NativeModules}.swift \
src/pythonnative/native/ios/Sources/PythonNativeKit/Generated/
cp build/contracts/{PNContracts,NativeProps,NativeModules}.kt \
src/pythonnative/native/android/src/main/java/com/pythonnative/generated/
uv run pytest tests/test_codegen.py
```

The test checks that the committed native contracts match generation in a clean
Python interpreter. The other files under `build/contracts/` are local outputs;
don't add the whole directory to the commit.

Exclude staged app projects, compiled apps and libraries, package distributions,
test reports, local storage, credentials, and tool caches. Examples include
`build/`, `dist/`, `.build/`, `.gradle/`, `.cxx/`, `.swiftpm/`, and `__pycache__/`.
The vendored browser runtime above is an explicit exception to excluding
compiled dependencies; downloaded Python runtimes and locally compiled binaries
aren't.

## Conventional Commits

This project uses Conventional Commits. Use the form:
Expand Down Expand Up @@ -154,7 +202,7 @@ Recommended scopes (choose the smallest, most accurate unit; prefer module/direc
- `devserver`: dev server, file watcher, and WebSocket implementation (`devserver/`)
- `devclient`: on-device dev client that syncs sources and Fast Refreshes (`devclient.py`)
- `hot_reload`: module reloader and Fast Refresh (`hot_reload.py`)
- `layout`: pure-Python flexbox engine (`layout.py`)
- `layout`: Yoga layout integration and the host binding (`layout.py`, `native/yoga/`)
- `mutations`: batched mutation ops between reconciler and native backends (`mutations.py`)
- `native_modules`: native module registry, Python facades, and Python fallbacks (`native_modules/`)
- `native_views`: view registry protocol and bridge backend (`native_views/`)
Expand Down Expand Up @@ -182,8 +230,8 @@ Recommended scopes (choose the smallest, most accurate unit; prefer module/direc
- `repo`: repository metadata and top‑level files (`README.md`, `CONTRIBUTING.md`, `.gitignore`, licenses)
- `scripts`: developer scripts under `scripts/` (e.g., `check.sh`)
- `templates`: Android/iOS project templates under `src/pythonnative/templates/`
- `kit`: the Swift rendering core (`templates/ios_template/PythonNativeKit/`)
- `runtime-android`: the Kotlin rendering core (`templates/android_template/pythonnative/`)
- `kit`: the Swift rendering core (`native/ios/`)
- `runtime-android`: the Kotlin rendering core (`native/android/`)
- `tests`: unit/integration/E2E tests under `tests/`
- `workflows`: CI pipelines under `.github/workflows/`

Expand Down Expand Up @@ -281,7 +329,8 @@ Co-authored-by: Name <email>
- Lint/format: `uv run ruff check .` and `uv run black --check src examples tests` pass.
- Docs: update `README.md` if behavior changes.
- Templates: update `src/pythonnative/templates/` if generator output changes.
- No generated artifacts committed.
- Generated contracts and example schemas are regenerated and reviewed when changed; vendored dependencies retain their licenses and provenance.
- No local build outputs, caches, credentials, or machine-specific files are committed.

## Versioning and releases

Expand Down Expand Up @@ -372,7 +421,7 @@ When you add a new public symbol you must also:
### CI

- **CI** (`ci.yml`): runs formatter, linter, type checker, and tests on every push and PR.
- **E2E** (`e2e.yml`): builds the hello-world example on Android (Linux emulator) and iOS (macOS simulator), then runs Maestro flows. Triggers on pushes to `main`, PRs, and manual dispatch.
- **E2E** (`e2e.yml`): builds the `e2e-suite` example on Android (Linux emulator) and iOS (macOS simulator), then runs Maestro flows. Triggers on pushes to `main`, PRs, and manual dispatch.
- **Packages** (`packages.yml`): resolves the PyPI compatibility matrix in `tests/packages/matrix.toml` against the live indexes with `scripts/package-matrix.py --check`, weekly and on changes to the resolver or manifest, and uploads the rendered Markdown table for `docs/guides/pypi-packages.md`.
- **PR Lint** (`pr-lint.yml`): validates the PR title against Conventional Commits format (protects squash merges) and checks individual commit messages via commitlint (protects rebase merges). Recommended: add the **PR title** job as a required status check in branch-protection settings.
- **Release** (`release.yml`): runs on merge to `main`; computes version, generates changelog, tags, creates GitHub Release, and (when `DRAFT_RELEASE` is `"false"`) publishes to PyPI.
Expand Down
11 changes: 11 additions & 0 deletions MANIFEST.in
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
global-exclude *.pyc *.pyo .DS_Store
recursive-exclude src/pythonnative/native .build/* .gradle/* .cxx/* build/* .swiftpm/* .kotlin/*
prune src/pythonnative/native/android/build
prune src/pythonnative/native/android/.gradle
prune src/pythonnative/native/android/.cxx
prune src/pythonnative/native/ios/.build
prune src/pythonnative/native/ios/.swiftpm
prune src/pythonnative/native/yoga/.build
prune src/pythonnative/templates/android_template/app/build
prune src/pythonnative/templates/android_template/.gradle
prune src/pythonnative/templates/ios_template/build
2 changes: 1 addition & 1 deletion docs/api/component-properties.md
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -477,7 +477,7 @@ pn.FlatList(data=items, render_item=render_fn, key_extractor=key_fn,
- `render_item`: `(item, index) -> Element` function
- `key_extractor`: `(item, index) -> str` for stable keys
- `item_height`: fixed row height; qualifies the list for the native
RecyclerView / UITableView virtualization path (see the
RecyclerView / UICollectionView virtualization path (see the
[Lists guide](../guides/lists.md))
- `separator_height`: spacing between items
- `horizontal`: lay rows out left-to-right (Python-windowed engine)
Expand Down
35 changes: 5 additions & 30 deletions docs/api/images.md
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
@@ -1,33 +1,8 @@
# Images

The shared image pipeline behind the
[`Image`][pythonnative.Image] and
[`ImageBackground`][pythonnative.ImageBackground] components. Remote
sources are downloaded on a background thread with an in-memory LRU
cache, a disk cache, and request deduplication (many views asking for
the same URL trigger one download). Platform handlers decode the
cached file downsampled to the view's bounds, so oversized photos
don't pin full-resolution bitmaps in memory.
`Image` loads and decodes images with the platform's native image loader. Each
platform caches responses and cancels obsolete requests when a source changes
or a component unmounts. Python doesn't download or decode image bytes.

Most apps never call this module directly; the `Image` component's
`placeholder_color`, `on_load`, and `on_error` props cover the common
cases:

```python
import pythonnative as pn

pn.Image(
source="https://example.com/photo.jpg",
placeholder_color="#E2E8F0",
on_load=lambda: print("loaded"),
on_error=lambda message: print("failed:", message),
style={"width": 200, "height": 120},
)
```

::: pythonnative.images
options:
show_root_heading: false
show_root_toc_entry: false
members_order: source
filters: ["!^_"]
Use `Image(source=...)` for bundled resources, files, or remote images. Supply
layout constraints when an image must reserve space before its content loads.
2 changes: 1 addition & 1 deletion docs/api/pythonnative.md
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ The reference is split per module so each page stays scannable:
| Testing | [Testing](testing.md) | [`render`][pythonnative.testing.render], [`render_hook`][pythonnative.testing.render_hook], [`FakeBackend`][pythonnative.testing.FakeBackend], [`FakeHost`][pythonnative.testing.FakeHost] |
| Styling | [Style](style.md) | [`StyleSheet`][pythonnative.StyleSheet], [`Style`][pythonnative.style.Style], [`StyleProp`][pythonnative.style.StyleProp], [`style`][pythonnative.style.style], [`Theme`][pythonnative.Theme], [`ThemeContext`][pythonnative.style.ThemeContext], [`use_theme`][pythonnative.use_theme] |
| Appearance | [Appearance](appearance.md) | [`use_color_scheme`][pythonnative.use_color_scheme], `appearance.set_color_scheme`, `appearance.get_color_scheme` |
| Images | [Images](images.md) | `images.fetch`, `images.clear_cache` |
| Images | [Images](images.md) | `Image`, `ImageBackground` |
| Element descriptor | [Element](element.md) | [`Element`][pythonnative.Element] |
| Screen host | [Hosts](hosts.md) | [`create_screen`][pythonnative.create_screen], [`ScreenHost`][pythonnative.hosts.base.ScreenHost] |
| Reconciler | [Reconciler](reconciler.md) | [`Reconciler`][pythonnative.reconciler.Reconciler] |
Expand Down
53 changes: 15 additions & 38 deletions docs/api/runtime.md
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
@@ -1,44 +1,21 @@
# Async runtime

PythonNative runs a single framework-wide ``asyncio`` event loop **on
the platform's main thread**, pumped as a guest of the native run loop
(``dispatch_async`` on iOS, ``Handler.post`` on Android, the
transport's main loop in ``pn preview``). Every awaitable surface in the framework
schedules its work on this loop: ``async def`` components,
coroutine [`use_effect`][pythonnative.use_effect] callbacks,
[`use_resource`][pythonnative.use_resource],
[`use_query`][pythonnative.hooks.use_query],
[`use_mutation`][pythonnative.hooks.use_mutation],
[`fetch`][pythonnative.net.fetch],
[`AsyncStorage`][pythonnative.storage.AsyncStorage], the awaitable native
modules ([`Camera`][pythonnative.native_modules.camera.Camera] /
[`Location`][pythonnative.native_modules.location.Location] /
[`Notifications`][pythonnative.native_modules.notifications.Notifications]),
and [`Animated`][pythonnative.animated.Animated] composites.
PythonNative starts one standard `asyncio` event loop on a dedicated application
thread. Component rendering, effects, event handlers, and async tasks run on
that thread. UIKit and Android retain their own UI threads; native bridge calls
marshal view operations there and queue events back to Python.

::: pythonnative.runtime
options:
show_root_heading: false
show_root_toc_entry: false
members_order: source
filters: ["!^_"]

## Pattern: bridge a sync handler into async code

```python
import pythonnative as pn
Ordinary asyncio networking, `TaskGroup`, timeouts, synchronization primitives,
and third-party async libraries work without a guest-loop adapter. A synchronous
Python callback can still delay other Python work, so use `asyncio.to_thread`
for blocking I/O and cooperative async work for long operations.

Component effects and async event handlers have component lifetimes. Unmounting
cancels their tasks. Use `runtime.run_application_task()` when work must survive
the initiating component, and `TaskScope` for explicitly owned services.
Native promises can register cancellation handlers; late results are ignored.

@pn.component
def Toolbar():
async def export():
report = await build_report()
await save_to_disk(report)
Headless tests can use `run_blocking()` and `drain()`. Don't call `run_blocking()`
from the running application loop; await the operation instead.

return pn.Button("Export", on_press=lambda: pn.run_async(export()))
```

## Next steps

- Walk through the async surface end-to-end:
[Async + data guide](../guides/async.md).
::: pythonnative.runtime
3 changes: 1 addition & 2 deletions docs/api/suspense.md
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@

Primitives behind PythonNative's async rendering model: the
[`Suspend`][pythonnative.suspense.Suspend] signal, the
[`CoroDriver`][pythonnative.suspense.CoroDriver] that steps `async def`
component bodies synchronously, cached async values
standard asyncio tasks that run `async def` component bodies, cached async values
([`Resource`][pythonnative.Resource] /
[`start_resource`][pythonnative.start_resource]), and code splitting
with [`lazy`][pythonnative.lazy].
Expand Down
Loading
Loading
Back | FazBrowse Home | New Git URL