What is OpenCompass ? OpenCompass is a platform focused on understanding of the AGI, include Large Language Model and Multi-modality Model.
We aim to:
- develop high-quality libraries to reduce the difficulties in evaluation
- provide convincing leaderboards for improving the understanding of the large models
- create powerful toolchains targeting a variety of abilities and tasks
- build solid benchmarks to support the large model research
- research on inference of Large Model(analysis, reasoning, prompt engineering.)
OpenCompass
- OpenCompass is an LLM evaluation platform, supporting a wide range of models (LLaMA, LLaMa2, ChatGLM2, ChatGPT, Claude, etc) over 80+ datasets.
- https://github.com/open-compass/opencompass
VLMEvalKit
- VLMEvalKit is a toolkit for evaluating large vision-language models (LVLMs), currently supporting ~20 LVLMs and five multi-modal benchmarks.
- https://github.com/open-compass/vlmevalkit
CompassVerifier
- CompassVerifier is an accurate and robust lightweight verifier model for evaluation and outcome reward.
- https://github.com/open-compass/CompassVerifier
CompassJudger
- CompassJudger-2: Towards Generalist Judge Model via Verifiable Rewards
- https://github.com/open-compass/CompassJudger
| Project | Topic | Paper |
| Automated Software Development | ||
| Critic Reasoning | ||
| Hallucination Annotation |
ANAH: Analytical Annotation of Hallucinations in Large Language Models |
|
| Mathematical Reasoning | ||
| Tool Utilization |
T-Eval: Evaluating the Tool Utilization Capability Step by Step |
|
| Multi Modality | ||
| Subjective Evaluation |
BotChat: Evaluating LLMs’ Capabilities of Having Multi-Turn Dialogues |
|
| Domain Evaluation |
LawBench: Benchmarking Legal Knowledge of Large Language Models |
Pinned Loading
Repositories
- opencompass Public
OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.
- VLMEvalKit Public
Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
- AgentCompass Public
AgentCompass is an extensible open-source evaluation infrastructure for systematically assessing LLM/VLM agent capabilities.
- MCPServer-AC Public
- docs Public
- PowerBench Public
- CNFinBench Public
CNFinBench — the first comprehensive benchmark for high-stakes financial scenarios. It spans 29 subtasks grounded in authoritative financial corpora and real business contexts, reconstructing end-to-end agent execution chains from requirement parsing, path planning, tool invocation, to result verification.
- GenEditEvalKit Public
The first unified, efficient, and extensible evaluation toolkit for evaluating image generation and editing models across multiple benchmarks.
- TextEdit Public
We provide TextEdit, a high-quality, multi-scenario text editing benchmark for generation models.
Top languages
Loading…
Most used topics
Loading…