If the template is not applicable for a particular example, the prompt.apply() method returns only a single-element list with an empty string instead of a two-element list with the prompted input and the reference answer. This contradicts the docstring which says that two elements are returned.
Should we just update the docstring, or handle this case with more care?
Example: the e2e_nlg_cleaned dataset with QA-style templates such as food_qa (1acabbc3-c9b9-4624-a684-29faeccff46f). This manifests in the web interface by displaying "∅∅∅ Blank result" instead of an input-target pair.
Reactions are currently unavailable
If the template is not applicable for a particular example, the prompt.apply() method returns only a single-element list with an empty string instead of a two-element list with the prompted input and the reference answer. This contradicts the docstring which says that two elements are returned.
Should we just update the docstring, or handle this case with more care?
Example: the e2e_nlg_cleaned dataset with QA-style templates such as food_qa (1acabbc3-c9b9-4624-a684-29faeccff46f). This manifests in the web interface by displaying "∅∅∅ Blank result" instead of an input-target pair.