AI45Lab/DeepSafe: All-in-One Safety Evaluation Framwork · GitHub

Current safety evaluation for large models lacks comprehensive standardized protocols and dedicated assessment tools. DeepSafe is the first all-in-one framework integrating 25+ safety datasets and the specialized ProGuard evaluation model, supporting full-modal LLM/VLM assessment.

DeepSafe is part of DeepSight and works best with 🔍 DeepScan (LLM/MLLM diagnosis toolkit). See the unified evaluation–diagnosis workflow on the HomePage.

• 2026-07-15: DeepSafe-Sci is now available on the deepsafe-sci. It provides dedicated evaluation workflows for SciHazard, Safe-Scientist, and SOSBench, covering harmful scientific assistance, safe handling of risky research requests, and over-refusal in scientific settings.
• 🔥🔥🔥 2026-02-06: DeepSafe's latest leaderboards and in-depth analyses are out now! Our evaluation comprehensively covers mainstream LLMs and MLLMs, including GPT, Claude, Gemini, DeepSeek, Qwen, Llama, and Mistral, providing a deep dive into the current state of AI safety. Access full results: Leaderboard | Frontier Risk Report | Content Risk Report

DeepSafe features a modular, configuration-driven elastic architecture, enabling a full-link automated closed loop from inference and generation to judgment and deep evaluation reporting. It provides a deeply evaluable, reproducible, and highly scalable evolving security infrastructure for AI Safety research, aiming to drive safety assessment from superficial testing to in-depth analysis and accelerate the construction of Trustworthy AI. 🚀

• High Extensibility: Powered by a Registry mechanism, new components (datasets, metrics, etc.) can be integrated with minimal code. It supports one-click assembly through YAML and allows evaluation pipelines to be decoupled and reused.
• Streamlined Usability: Adopts a "Config-as-Execution" paradigm. Simply provide a single YAML file, and the framework automatically completes the full cycle and generates standardized reports.
• Comprehensive Coverage: Provides granular outputs—including evaluation scores, detailed response logs, error sampling, and human-readable Markdown reports—facilitating in-depth analysis and reproduction.

• Proactive Risk Identification: Introduces a pioneering proactive detection paradigm capable of reasoning about and describing unknown risks, transcending the rigid constraints of predefined classification systems.
• Eradicating Modality Bias: Implements a hierarchical multimodal safety taxonomy trained on a balanced dataset of 87,000 samples, ensuring fair and precise risk assessment across both text and visual modalities.

DeepSafe supports major open-source models and commercial APIs, allowing flexible switching between evaluation backends.

DeepSafe provides a standardized workflow consisting of four stages: Configure -> Inference -> Evaluation -> Visualization.

# Recommended to use a virtual environment
pip install -r requirements.txt
# huggingface-cli is required for downloading datasets
pip install -U huggingface_hub

# Verify if the environment is set up correctly
python smoke_test.py

Most datasets supported by DeepSafe are hosted on Hugging Face. You can use huggingface-cli to download them.

Download Example (e.g., Do-Not-Answer):

# Download dataset to a local directory
huggingface-cli download --repo-type dataset --resume-download LibrAI/do-not-answer --local-dir data/do-not-answer --local-dir-use-symlinks False

After downloading, please update the dataset.path field in the corresponding YAML configuration file (e.g., configs/eval_tasks/do_not_answer_v01.yaml) to point to your local path.

The Salad-Bench dataset needs to be manually downloaded from the official repository. You can use the following command:

# Download the dataset
huggingface-cli download --repo-type dataset --resume-download OpenSafetyLab/Salad-Data --local-dir Salad-Data --local-dir-use-symlinks False

Then move or copy base_set.json to your specified local directory. Fill in your local path in the dataset.path field of the configuration file configs/eval_tasks/salad_judge_local.yaml:

dataset:
  type: SaladDataset
  path: /your/path/Salad-Data/base_set.json

mdjudge (MD-Judge-v0.1) is the safety evaluator officially recommended by Salad-Bench. The model weights can be downloaded via HuggingFace:

# Download safetensors weight files and config files
huggingface-cli download --resume-download OpenSafetyLab/MD-Judge-v0.1 --local-dir MD-Judge-v0.1 --local-dir-use-symlinks False

Place the MD-Judge-v0.1 folder in any specified local directory. Then, in the evaluator.judge_model_cfg.model_name field, replace it with the local path, for example:

evaluator:
  judge_model_cfg:
    type: VLLMLocalModel
    model_name: /your/local/dir/MD-Judge-v0.1
    # Other configurations remain unchanged

Note: For the first-time use, ensure the evaluation machine can load the weight files, and the local configuration path matches the actual download path; otherwise, the evaluation will fail to find the model.

dataset:
  type: SaladDataset
  path: /your/path/to/Salad-Data/base_set_sample_100.json

evaluator:
  type: ScorerBasedEvaluator
  batch_size: 32
  template_name: md_judge_v0_1
  judge_model_cfg:
    type: VLLMLocalModel
    model_name: /your/path/to/MD-Judge-v0.1
    tensor_parallel_size: 1
    gpu_memory_utilization: 0.4
    trust_remote_code: true
    temperature: 0.0
    max_tokens: 64

If there are other dependencies or execution errors, please verify the config paths and model formats.

DeepSafe provides one-shot local scripts. Simply run:

# Execute from the project root
bash scripts/run_salad_local.sh configs/eval_tasks/salad_judge_local.yaml

• Configure: Specify target model, dataset paths, and judge model parameters in the YAML file.
• Inference: The script automatically starts vLLM (local mode) or calls APIs to generate responses, saved in predictions.jsonl.
• Evaluation: Launches ProGuard or other judge models to automate scoring and categorization.
• Visualization: Summarizes metrics and generates a human-readable report.md in the output directory.

Parameters explained using salad_judge_v01_qwen1.5-0.5b_vllm_local.yaml:

Integrate your own dataset in three simple steps:

Ensure your data file follows this JSONL structure: {"id": "001", "prompt": "User Input", "reference": "Ground Truth", "category": "Label"}

• Dataset (uni_eval/datasets/): Inherit BaseDataset and override load() to read your JSONL.
• Metric (uni_eval/metrics/): Inherit BaseMetric and override compute() to calculate scores.
• Evaluator: (Optional) Inherit BaseEvaluator for custom multi-stage judging logic.

Register your modules in the respective __init__.py files, create your YAML config, and you are ready!

DeepSafe/
├── uni_eval/                # Core Evaluation Framework
│   ├── datasets/            # Dataset Loader Implementations
│   ├── models/              # Model Adapters (API/HF/vLLM)
│   ├── evaluators/          # Evaluation Logic Controllers
│   ├── metrics/             # Evaluation Metric Implementations
│   ├── runners/             # Task Execution Management
│   ├── summarizers/         # Result Summarization and Reporting
│   ├── cli/                 # CLI Parsing Tools
│   └── registry.py          # Registry Center
├── configs/                 # YAML Configuration Files
├── scripts/                 # Launch Scripts (Shell)
├── tools/                   # Utility Scripts
└── results/                 # Evaluation Outputs (JSON/Markdown)

Contributions are welcome! Please submit an Issue or Pull Request to help us build DeepSafe. 🌟

• Email: shaojing@pjlab.org.cn