VisionXLab/FIRM-Reward: Official repository for the FIRM Reward series · GitHub

[Image Generation & Edit] Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation
[Video Generation] FIRM-Video: Check Before You Score for Reliable Text-to-Video Reward Modeling

• Critics are the bottleneck. The FIRM series is built around a simple belief: RL for visual generation only works when the reward model is faithful, stable, and hard to hack.
• Three task-specific data pipelines. FIRM-Edit uses a difference-first scoring pipeline, FIRM-Gen uses a plan-then-score pipeline to reduce MLLM hallucinations, and FIRM-Video uses a check-before-score pipeline to ground video scores in verified evidence.
• Benchmarks for critic reliability. FIRM-Bench provides human-annotated evaluation for editing and generation critics, while FIRM-Video-Bench extends this evaluation to video reward models.
• Reward shaping that actually holds up. CME and QMA are designed to prevent the shortcut behavior that appears when rewards are naively combined.

TrustYourCritic/
├── generation/   # Image Generation RL training and reward serving
└── editing/      # Image Edit RL training, reward serving, reproduction scripts
└── video_generarion/   # Reserved for our planned Video Generation RL code release — stay tuned

• To avoid Python package conflicts, install and run GenRL/EditRL in separate environments.

cd generation
conda create -n FIRM-Gen python=3.10 -y
conda activate FIRM-Gen
pip install -e .

python generation/flow_grpo/reward_model_server.py

• generation/config/nft_flux2_klein.py
• generation/config/nft_qwen_image.py
• generation/config/nft_zimage_turbo.py
• generation/config/nft.py

bash generation/scripts/train_sd35_sharegpt_qwenvl.sh

cd editing
conda create -n FIRM-Edit python=3.10 -y
conda activate FIRM-Edit
pip install -e .

## Change the default ip and port to your perference
python editing/reward_server/reward_server_qwen3_vl_8b_sft.py

• editing/config/kontext_nft_qwen3vl_8b_sft.py
• editing/config/kontext_nft_qwen3vl_8b.py
• editing/config/kontext_nft_qwen25vl_32b_non_logits.py

bash editing/examples/train_qwen_image_edit.sh

Coming soon.

Expected JSON file like:

[
  {"input_prompt": "A cinematic portrait of a fox in snow."}
]

Expected dataset layout:

dataset-root/
├── images/
├── train_metadata.jsonl
└── test_metadata.jsonl

Each JSONL line:

{"prompt": "make the sky sunset orange", "image": "images/example.jpg", "requirement": "preserve identity"}

Coming soon.

The code and data for FIRM-Bench&FIRM-Bench-Video are hosted on Hugging Face.

We provide inference and evaluation scripts for FIRM-Bench&FIRM-Bench-Video. We recommend deploying the model with vLLM for inference.

python FIRM-Bench-Gen/vllm_infer.py \
  --input FIRM-Bench-Gen/bench_v1.jsonl \
  --output FIRM-Bench-Gen/result/xxx.jsonl \
  --image-root FIRM-Bench-Gen/ \
  --api-url xxxxx

python FIRM-Bench-Gen/gen_mae.py \
  --gt FIRM-Bench-Gen/result/human_bench_v1.jsonl \
  --pred FIRM-Bench-Gen/result/xxx.jsonl

python FIRM-Bench-Edit/vllm_infer.py \
  --input FIRM-Bench-Edit/bench_v1.jsonl \
  --output FIRM-Bench-Edit/result/xxx.jsonl \
  --image-root FIRM-Bench-Edit/ \
  --api-url xxxxx

python FIRM-Bench-Edit/edit_mae.py \
  --gt FIRM-Bench-Edit/result/human_bench_v1.jsonl \
  --pred FIRM-Bench-Edit/result/xxx.jsonl

bash scripts/infer.sh all 32 qwen3vl8b \
  data/firm-video-bench.json \
  http://127.0.0.1:8000/v1 \
  Qwen3-VL-8B-Instruct

python scripts/metric.py \
  --gt_file data/firm-video-bench.json \
  --results_dir results

This repository was shaped by several open-source projects that pushed RL for image generation and image editing forward:

• flow-grpo
• DiffusionNFT
• Edit-R1

If you find this project, models, datasets, or benchmarks useful in your research, please cite the corresponding papers:

@misc{zhao2026trustcriticrobustreward,
  title         = {Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation},
  author        = {Xiangyu Zhao and Peiyuan Zhang and Junming Lin and Tianhao Liang and Yuchen Duan and Shengyuan Ding and Changyao Tian and Yuhang Zang and Junchi Yan and Xue Yang},
  year          = {2026},
  eprint        = {2603.12247},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2603.12247}
}

@misc{zhang2026firmvideocheckscorereliable,
  title         = {FIRM-Video: Check Before You Score for Reliable Text-to-Video Reward Modeling},
  author        = {Peiyuan Zhang and Xiangyu Zhao and Hongbo Liu and Xiaoxing Hu and Mingxin Liu and Shuran Ma and Yunhang Shen and Jian Hu and Haihan Gao and Haoyu Cao and Xue Yang},
  year          = {2026},
  eprint        = {2608.21839},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2608.21839}
}