Loading module from “http://localhost:8000/dist/core.js” was blocked because of a disallowed MIME type (“”)

that's out of tree because you start mini-coi with tests/ as root ... npx mini-coi -p 8000 ./ is all you need then reach the http://localhost:8000/tests/ URL

there's a fundamental misconception of how dependencies work in PyScript and its used runtimes ... indeed:

Please read my comments in this MR to change mpy to py and explicitly include dependencies per each worker, thanks.

there's a fundamental misconception of how dependencies work in PyScript and its used runtimes ... indeed:

Then, when I tried this approach in pyscript.com, this error is the JSON it's failing to load? I used only Pyodide there.

Please read my comments in this MR to change mpy to py and explicitly include dependencies per each worker, thanks.

Well, whatever, I'll switch to Pyodide workers for the time being, and add another, similar micropython test later.

I can run the pure-Python tests now! I guess it's just a documentation issue, and a regrettably misleading error message.

However...

================================= FAILURES =================================
Failed: ./tests/test_workers.py::test_find_path_networkx_parallel_pyodide
Traceback (most recent call last):
  File "upytest.py", line 182, in run
  File "tests/test_workers.py", line 120, in test_find_path_networkx_parallel_pyodide
ImportError: can't import name Random

I put in an explicit dependency on the random stdlib package in settings_py.json to be safe. No dice.

I'm afraid https://packages.pyscript.net/package/?package=random is not available ??? is that a native Python thing? if so, shouldn't it be random (lowercase) instead?

to clarify, non Python stdlib things should be in packages, packages that are in stdlib should not be part of packages ... they are not dependencies, they are core.

we are here: https://pyscript.com/@agiammarchi/parallel-pathfinding-bare-coroutines-str-copy/latest?files=main.py,worker.py,pyscript.toml,index.html

You need to pass config to each worker ... each worker is a runtime/world a part ... but now there is another error, although all imports are just fine, all workers run, everything is fine except the code that is expecting something else ... not sure what.

I'm afraid https://packages.pyscript.net/package/?package=random is not available ??? is that a native Python thing? if so, shouldn't it be random (lowercase) instead?

random.Random is the randomizer class, which you can instantiate if you want to have your own pseudorandomizer with a different state from the global one in the random module. I imported it as from random import Random, which should work, but doesn't.

Oh, I think we may have stumbled into a compatibility issue between PyScript and Firefox, as well, because the output of your code on Firefox is different:

(PY0409) JSON.parse: unexpected character at line 1 column 8 of the JSON data for <script type="py" src="./worker.py" config="./pyscript.toml" worker="" name="worker0"></script>(PY0409) JSON.parse: unexpected character at line 1 column 8 of the JSON data for <script type="py" src="./worker.py" config="./pyscript.toml" worker="" name="worker2"></script>(PY0409) JSON.parse: unexpected character at line 1 column 8 of the JSON data for <script type="py" src="./worker.py" config="./pyscript.toml" worker="" name="worker1"></script>(PY0409) JSON.parse: unexpected character at line 1 column 8 of the JSON data for <script type="py" src="./worker.py" config="./pyscript.toml" worker="" name="worker3"></script>generated barbell_graph
here we go
1. we want a path 3→2
we submitted a request to find 3→2
2. we want a path 5→4
we submitted a request to find 5→4
3. we want a path 0→1
we submitted a request to find 0→1
4. we want a path 6→7
we submitted a request to find 6→7

And then it hangs forever.

And I'm also getting different output from you when I fork your fork of parallel-pathfinding in Chromium!

(PY0409) Unexpected token ']', "[ "s1",]" is not valid JSON for <script type="py" src="./worker.py" config="./pyscript.toml" worker="" name="worker0"></script>(PY0409) Unexpected token ']', "[ "s1",]" is not valid JSON for <script type="py" src="./worker.py" config="./pyscript.toml" worker="" name="worker1"></script>(PY0409) Unexpected token ']', "[ "s1",]" is not valid JSON for <script type="py" src="./worker.py" config="./pyscript.toml" worker="" name="worker2"></script>(PY0409) Unexpected token ']', "[ "s1",]" is not valid JSON for <script type="py" src="./worker.py" config="./pyscript.toml" worker="" name="worker3"></script>generated barbell_graph
here we go
1. we want a path 5→7
we submitted a request to find 5→7
2. we want a path 6→3
we submitted a request to find 6→3
3. we want a path 9→0
we submitted a request to find 9→0
4. we want a path 4→2
we submitted a request to find 4→2
5. we want a path 1→8
we submitted a request to find 1→8

It's the Linux Mint build of Chromium, which has some customizations, I suppose...

I think I've gotten networkx to install in micropython? At least, now the test hangs, rather than giving me an error in mip...

When I forked your project on pyscript.com, it resulted in this project that, when run, gives me different output, though I haven't changed the code at all. Not sure what that's about.

I think I've gotten networkx to install in micropython? At least, now the test hangs, rather than giving me an error in mip...

Oh, it didn't hang! It just took longer than I expected to grab all those files. Here's the new failure I'm looking at:

Failed: ./tests/test_workers.py::test_find_path_networkx_parallel_pyodide
Traceback (most recent call last):
  File "upytest.py", line 182, in run
  File "tests/test_workers.py", line 124, in test_find_path_networkx_parallel_pyodide
  File "/lib/networkx/__init__.py", line 15, in <module>
  File "/lib/networkx/lazy_imports.py", line 1, in <module>
ImportError: no module named 'importlib'

I probably have to add that dependency in the package.json somehow...or patch networkx not to need it?

There was an issue with my test data generation code. I fixed it by running the test function outside of upytest, with plentiful prints.

Now I'm stuck here

        coros = []
        nodepairs = []
        # then submit nodes for them to find paths between
        for worker in our_workers:
            a = nodes.pop()

            nodes.insert(0, a)
            b = nodes.pop()

            nodes.insert(0, b)
            nodepairs.append((a, b))

            coros.append(worker.dijkstra_path(graph_d, a, b))
        for coro, (a, b), expected in zip(coros, nodepairs, expectations[name]):
            print(f"will await {coro} to find path {a}->{b}")
            the_path = await coro
            print(f"{coro} got path {the_path}")

            assert the_path == expected, (
                f"The path from {a} to {b} in {name} should be {expected}; instead, got {the_path}"
            )

It's the last part of test_find_path_parallel, and none of those coroutine-like objects ever seem to yield anything. I've run the exact same code in the main thread by this point. It generated the test data.

Where do I go from here?

I have verified that code is at least running in the workers. They're having some trouble returning the result.

every worker bootstraps a new Pyodide environment ... meaning, every worker will add 1 up to 3+ seconds to bootstrap, depending on the config.

I don't know if this is anyhow meaningful, but this example is not minimal, it's complex, and the more complex it gets, the least "easy-to-solve" becomes ... can you reduce your test case to a single entity and tell us if that still fails?

in case it's not clear, if we have a single line example (that's enough to track issues) that shows what is failing, we might help ... if you keep changing tests and this PR because your tests were failing or anything, that does not help us.

unfortunately our time to tackle issues are extremely tight, so that every update I read about this topic to me is like "see? good thing you haven't wasted time around that issue yet".

I want a stable, consistent, precise thing I can read with ease and replicate/debug, otherwise this can keep going for a very long time without resolution ... I want to solve this, if there's anything we're doing wrong, I am not playing the "long time bug keeps getting updates around its tests" game, with all due respect I want to appreciate in here around your effort around this topic too!

Thanks for your understanding!

There's something preventing me from sending a graph's worth of data to a worker. I don't think the data is especially large.

https://pyscript.com/@clayote/cheating-dijkstra/latest

Traceback (most recent call last):
  File "/lib/python313.zip/_pyodide/_base.py", line 597, in eval_code_async
    await CodeRunner(
    ...<9 lines>...
    .run_async(globals, locals)
  File "/lib/python313.zip/_pyodide/_base.py", line 413, in run_async
    await coroutine
  File "<exec>", line 11, in <module>
pyodide.ffi.JsException: Error: This borrowed proxy was automatically destroyed when an iterator was exhausted.
For more information about the cause of this error, use `pyodide.setDebug(true)

The pathfinding pool tests all pass now. Some of the others in the same file have regressed, but I didn't change them, apart from running ruff format.

This should probably be a squash merge.

The pathfinding pool tests all pass now. Some of the others in the same file have regressed, but I didn't change them, apart from running ruff format.

This was not actually true. I deleted the worker functions the old math tests used, for reasons I no longer remember. I've put them back.

I realize I have little credibility left, but these tests pass on Chromium and Firefox.

@clayote gonna be honest, I am not sure what we're trying to achieve in here or how important this is ... considering how many changes you made, I am not willing (mostly because I have zero extra capacity) to maintain so many changes for an example so I suggest you put your demo on pyscript.com and you have full ownership of its latest state? If that's really convincing we could feature it in our https://pyscript.net/community/ dedicated page but please bear with me I have 99 problems to solve right now and checking examples is not really a priority to me, I am afraid.