Search results

OpenAI-compat: --backend openai --base-url /v1, not default vllm or origin-only

Status: Open.

#315 In evalplus/evalplus;

· cursor opened on Aug 28, 2026

Optional EvalPort interop for HumanEval+/MBPP+ problems and eval results

Status: Open.

#314 In evalplus/evalplus;

· adhabnr-ux opened on Aug 22, 2026

Cross-project discussion: benchmark validity, reproducibility and provenance

Status: Open.

#312 In evalplus/evalplus;

· KeilerHirsch opened on Aug 14, 2026

Question: should Mbpp/284's plus tests discriminate == from is independently of string interning?

Status: Open.

#309 In evalplus/evalplus;

· toffee-desuwa opened on Jul 14, 2026

🐛 [TestRemoval] - Mbpp/287 float ground truth from canonical true division mismatches the documented integer task for 17 large-n tests

bugSomething isn't workingSomething isn't working

Status: Open.

#308 In evalplus/evalplus;

· toffee-desuwa opened on Jul 14, 2026

evalplus 0.3.1 on Windows: default config returns ('timeout', []) for every program — a silent false all-clear

Status: Open.

#307 In evalplus/evalplus;

· toffee-desuwa opened on Jul 14, 2026

🤗 [Leaderboard Submission] NemoSmith-4B-Instruct (Nemotron-4B-Instruct + RL Imitation for compilers) HumanEval+ & MBPP+

model eval

Status: Open.

#305 In evalplus/evalplus;

· abhilash1910 opened on Jul 9, 2026

Independent Model Evaluation Request: TemuClaude (OpenAI-compatible API)

Status: Open.

#304 In evalplus/evalplus;

· notSaiful opened on Jul 7, 2026

[REQUEST] - Inthasap-Council-Trinity

model eval

Status: Open.

#302 In evalplus/evalplus;

· hotmakthai opened on Jul 4, 2026

🐛 [TestRemoval] - <Mbpp/102> <Expect camelCase but test list expects PascalCase>

bugSomething isn't workingSomething isn't working

Status: Open.

#301 In evalplus/evalplus;

· Bcwz opened on Jun 29, 2026

[Leaderboard Submission] Gemma 4 31B via Google AI Studio (zero-shot) | HumanEval+ 91.5%

Status: Open.

#300 In evalplus/evalplus;

· hypernab opened on Apr 17, 2026

[Leaderboard Submission] Qwen3.6 35B A3B (Local Q4_K_M) HumanEval+ & MBPP+

Status: Open.

#299 In evalplus/evalplus;

· Iipal opened on Apr 17, 2026