Repository navigation
[WASI] test cases with sys.path.insert(0, os.curdir) fail on WASI #93461
Description
Bug report
Test cases that insert os.curdir ('.') into sys.path fail on WASI platforms. I think it is related how wasmtime maps directories into the runtime environment and how it sets up the current working directory. The test passes if I replace '.' with an empty string '' or remove the cached FileFinder instance for '.' with sys.path_importer_cache.pop('.').
./test-wasi.sh test_support -m test_forget
0:00:00 Run tests sequentially
0:00:00 [1/1] test_support
test test_support failed -- Traceback (most recent call last):
File "/Lib/test/test_support.py", line 116, in test_forget
mod = __import__(TESTFN)
^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '@test_66600_tmpæ'
test_support failed (1 error)
== Tests result: FAILURE ==
The culprit seems to be an outdated FileFinder instance: {'.': FileFinder('/.')}. The CWD is '/python-wasm/cpython/builddir/wasi/build/test_python_66600æ' when the test case is executed.
0:00:00 [1/1] test_support
> /Lib/test/test_support.py(114)test_forget()
-> sys.path.insert(0, os.curdir)
(Pdb) p sys.path
['.', '/usr/local/lib/python312.zip', '/Lib', '/build/lib.wasi-wasm32-3.12']
(Pdb) pp sys.path_importer_cache
{'.': FileFinder('/.'),
'/Lib': FileFinder('/Lib'),
'/Lib/collections': FileFinder('/Lib/collections'),
'/Lib/encodings': FileFinder('/Lib/encodings'),
'/Lib/importlib': FileFinder('/Lib/importlib'),
'/Lib/re': FileFinder('/Lib/re'),
'/Lib/test': FileFinder('/Lib/test'),
'/Lib/test/libregrtest': FileFinder('/Lib/test/libregrtest'),
'/Lib/test/support': FileFinder('/Lib/test/support'),
'/Lib/unittest': FileFinder('/Lib/unittest'),
'/Lib/urllib': FileFinder('/Lib/urllib'),
'/build/lib.wasi-wasm32-3.12': FileFinder('/build/lib.wasi-wasm32-3.12'),
'/usr/local/lib/python312.zip': None}
(Pdb) p os.getcwd()
'/python-wasm/cpython/builddir/wasi/build/test_python_66600æ'
(Pdb) n
> /Lib/test/test_support.py(115)test_forget()
-> importlib.invalidate_caches()
(Pdb) n
> /Lib/test/test_support.py(116)test_forget()
-> try:
(Pdb) p sys.path
['.', '.', '/usr/local/lib/python312.zip', '/Lib', '/build/lib.wasi-wasm32-3.12']
(Pdb) pp sys.path_importer_cache
{'.': FileFinder('/.'),
'/Lib': FileFinder('/Lib'),
'/Lib/collections': FileFinder('/Lib/collections'),
'/Lib/encodings': FileFinder('/Lib/encodings'),
'/Lib/importlib': FileFinder('/Lib/importlib'),
'/Lib/re': FileFinder('/Lib/re'),
'/Lib/test': FileFinder('/Lib/test'),
'/Lib/test/libregrtest': FileFinder('/Lib/test/libregrtest'),
'/Lib/test/support': FileFinder('/Lib/test/support'),
'/Lib/unittest': FileFinder('/Lib/unittest'),
'/Lib/urllib': FileFinder('/Lib/urllib'),
'/build/lib.wasi-wasm32-3.12': FileFinder('/build/lib.wasi-wasm32-3.12'),
'/usr/local/lib/python312.zip': None}
A build on Linux has an absolute path to build directory as first sys.path entry.
reproducer
$ podman run --rm -ti -v $(pwd):/python-wasm/cpython:Z quay.io/tiran/cpythonbuild:emsdk3 # ./build-python-build.sh # ./build-python-wasi.sh # ./test-wasi.sh test_support -m test_forget
Activity
pythongh-93461: Invalidate sys.path_importer_cache entries with relat…
tiran commented on Jun 9, 2022
GH-93461 is the simplest solution I could come up with. importlib.invalidate_caches now drops all FileFinder instances if the cache key is not an absolute path.
brettcannon commented on Jun 9, 2022
Seems like a reasonable solution to the problem.
gh-93461: Invalidate sys.path_importer_cache entries with relative pa…
pythongh-93461: Invalidate sys.path_importer_cache entries with relat…
gh-93461: Invalidate sys.path_importer_cache entries with relative pa…
brettcannon commented on Jun 17, 2022
@tiran can this be closed?
tiran commented on Jun 17, 2022
Yes, it is fixed now.
Revert 30935 defer bpo45162 to 312 (#94285)