Running into
{"status_code":400,"error":"Input schema validation failed: \"evaluationCriteria\" is required When reporting an issue add the request_id:'040ed360-593d-9316-b99a-142b41d3f4c1'"}

when setting up a test through the API with

client.validate.create_scenario_test("new baseline test images", slice_id="slc_c81fxadwzftg238jb5zg", evaluation_functions=[client.validate.eval_functions.bbox_iou()])

I guess this is some Pydantic issue, right?

If you have difficulty testing this PR (getting 400 errors), you are probably hitting the production scaleapi server instead of the local feature branch.

you would initialize the client like this: client = nucleus.NucleusClient(os.environ.get("API_KEY"), endpoint='http://localhost:3000/v1/nucleus') where on localhost:3000, you have the other feature branch checked out

To run pytest against the feature branch,

do: export NUCLEUS_ENDPOINT='http://localhost:3000/v1/nucleus in the window where you're running pytest

We also need to update the scenario_test.add_criterion(...) method.

I tried st.add_criterion(client.validate.eval_functions.bbox_map()) which doesn't work.

Let's also rename it to add_eval_function(...) in order to avoid confusion.

Renamed add_criterion to add_eval_function

Tested locally and interaction with the backend works!

My bad, forgot this earlier, let's also rename metric.get_criteria() to metric.get_eval_functions() to make it consistent.