This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Always read and follow the coding conventions in `~/.opencode/AGENTS.md`.
## What This Is
A **Pharo Smalltalk** project (not Python) that provides a Famix AST representation for Python code, built on TreeSitter. Part of the [moosetechnology/FAST](https://github.com/moosetechnology/FAST) ecosystem.
Source code format: **Tonel** (all source in `src/`).
## Build & Test
This is a Pharo Smalltalk project. There are no npm/pip/make commands.
CI is defined in `.github/workflows/tests.yml`. Tests run via `smalltalkci` using the `smalltalk.ston` spec. Coverage is collected in lcov format for `FAST-Python.*` packages.
There is no local test runner script outside of `smalltalkci`. You need a Pharo image with the project loaded to run tests interactively.
## Package Structure
```
src/
BaselineOfFASTPython/ -- Metacello baseline (dependency & group definitions)
`FAST-Python-Model/` contains ~130 generated class files (`FASTPy*`). These are produced by `FASTPythonMetamodelGenerator` in `FAST-Python-Model-Generator/`. **Do not edit model files directly** -- edit the generator and re-run it. The generator also produces a visitor trait (`FASTPyTVisitor`).
## Key Entry Points
| Class | Role |
|-------|------|
| `FASTPythonImporter` | Parse Python source string or file → FAST model. Extends `TSFASTAbstractImporter`. |
| `FASTPythonTreeSitterVisitor` | Walks TreeSitter CST → builds FAST model. 1100+ lines, the core import logic. |
| `FASTPythonCFGVisitor` | Builds control flow graph. Uses `FASTTCFGUtility` trait from FAST. |
| `FASTPythonLocalResolverVisitor` | Links entity usages to their local declarations (scope-aware). |
| `FASTPythonSSAVisitor` | SSA transform. Uses `FASTCFGTVisitor` trait from FAST. Requires local resolution first. |
| `FASTPythonMetamodelGenerator` | Generates the Famix metamodel. Run via `FASTPythonMetamodelGenerator new generate`. |
**Typical analysis pipeline:**
```smalltalk
model := FASTPythonImporter parseFile: aFile.
FASTPythonLocalResolverVisitor resolve: model module.
model allFunctionDefinitions first cfg. "CFG"
FASTPythonSSAVisitor resolve: model allFunctionDefinitions first. "SSA (after resolution)"
```
## Testing
- **Base test class**: `FASTPythonAbstractTestCase` (in Tools-Tests) -- provides a `parse:` helper that wraps `FASTPythonImporter` with error reporting.
- **Importer tests**: `FASTPythonImporterTest` extends `TSFASTAbstractImporterTest` (from TreeSitter). The test file is ~53k lines -- generated tests for every AST node type. Test generation snippet is in the class comment.
- **TreeSitter python grammar version matters**: After v0.25, expression statements were removed from the tree. If tests fail unexpectedly, check your `tree-sitter-python` version first (see README).
- **Pharo 13 is the primary target**; Pharo 14 is also tested. The baseline references `FAST:v3` which requires Moose 13.
- **The importer uses the master branch** of `TSLibrariesPython` (hardcoded in `FASTPythonTreeSitterVisitor class >> initialize`).
- **`expression --|> statement`**: In Python, expressions can be expression statements. This is reflected in the model hierarchy and affects how the visitor processes nodes.
- **CFG uses `FASTTCFGUtility`** from FAST (the parent project). SSA uses `FASTCFGTVisitor`. These traits define the core graph traversal protocol.
- **Local resolver scope management**: The resolver maintains a `scopes` stack. Shadowing is handled by checking entity kind consistency (same kind = shared declaration, different kind = new declaration).
- **Python 3 only for resolver**: The local resolver is Python 3 scoped. Python 2 comprehension scoping (no scope) is not supported (issue #26).
- **Non-local declarations are not deduplicated**: Each access to an unknown field creates a separate `FASTNonLocalDeclaration` (issue #26 in the repo).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- [TreeSitter python version compatibility](#treesitter-python-version-compatibility)
- [Python version compatibility](#python-version-compatibility)
- [Contact](#contact)
<!-- /TOC -->
## Installation
To install this project on your Pharo image, execute the following script:
Expand Down
Expand Up
@@ -47,6 +63,8 @@ Or
The best documentation to read about this project is located in Pharo Tree Sitter's repository here: [https://github.com/Evref-BL/Pharo-Tree-Sitter/blob/main/resources/doc/fast_importer.md](https://github.com/Evref-BL/Pharo-Tree-Sitter/blob/main/resources/doc/fast_importer.md) and here: [https://github.com/Evref-BL/Pharo-Tree-Sitter/blob/main/resources/doc/ts_utilities.md](https://github.com/Evref-BL/Pharo-Tree-Sitter/blob/main/resources/doc/ts_utilities.md)
You can find the documentation on some analysis possible here: [Analysis documentation](resources/doc/analysis.md).
## Control flow graph
Expand All
@@ -64,6 +82,35 @@ A CFG can be done on a function, method, class, module or lambda.
You can visualize it in the inspector as a visualization and you can also export your CFG as a mermaid visualization using `#asMermaidScript`
For more information check the analysis documentation linked above.
## Local resolutions
It is possible to apply a local name resolution on a FAST-Python model by executing this piece of code:
```smalltalk
FASTPythonLocalResolverVisitor resolve: model module "Can be any behavioral entity"
```
Once this is executed, all nodes resolved will be able to provide a `#localDeclaration` pointing to the declaration/first use of the entity. This local declaration also knows all the `#localUses` of the entity.
For more information check the analysis documentation linked above.
## Single Static Assignment (SSA)
It is possible to build the SSA of a FAST-Python model like this:
It can be run like this:
```smalltalk
FASTPythonSSAVisitor resolve: model module
```
Then we can ask to any node representing a variable access `#ssaVersion`. This version is able to tell us where are the read and write accesses of this particular version of the variable.
For more information check the analysis documentation linked above.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add local resolve and SSA #46
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Uh oh!
There was an error while loading. Please reload this page.
Add local resolve and SSA #46
Filter by extension
Deleted files Viewed files
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
There are no files selected for viewing
Uh oh!
There was an error while loading. Please reload this page.