EvalPlus version
0.4.0.dev38
Output of running ls ~/.cache/evalplus
N/A
Task ID of the programming task
Mbpp/102
Test input
[
"assert snake_to_camel('python_program')=='PythonProgram'",
"assert snake_to_camel('python_language')==('PythonLanguage')",
"assert snake_to_camel('programming_language')==('ProgrammingLanguage')"
]
modify to
[
"assert snake_to_camel('python_program')=='pythonProgram'",
"assert snake_to_camel('python_language')==('pythonLanguage')",
"assert snake_to_camel('programming_language')==('programmingLanguage')"
]Description
Referring to
#210 (comment)
https://huggingface.co/datasets/evalplus/mbppplus/viewer/default/test?row=55
The Test List is still asserting to PascalCase instead of camelCase, resulting in inconsistent evaluation results.
Other context
No response
Reactions are currently unavailable
EvalPlus version
0.4.0.dev38
Output of running ls ~/.cache/evalplus
N/A
Task ID of the programming task
Mbpp/102
Test input
Description
Referring to
#210 (comment)
https://huggingface.co/datasets/evalplus/mbppplus/viewer/default/test?row=55
The Test List is still asserting to PascalCase instead of camelCase, resulting in inconsistent evaluation results.
Other context
No response