🎯 Entropy-Adaptive Fine-Tuning :
Resolving Confident Conflicts to Mitigate Forgetting

If you like our project, please give us a star ⭐ on GitHub for the latest update.

• 📣 Latest News
• 📖 Abstract
• ⚡ Method: EAFT
• 📊 Results
• 🚀 Quick Start
• 📝 Citation

• [Jan 12,2026] 💻 We have implemented EAFT loss in a ms-swift-EAFT project!
• [Jan 08,2026] 🔥 We are honored to be featured as 🤗 HuggingFace Daily Paper #1.
• [Jan 07,2026] 📄 Our paper is now available on arXiv and Hugging Face daily paper.
• [Jan 06,2026] ✨ Integration: EAFT has been merged into LLaMA-Factory! You can now use EAFT via use_eaft_loss parameter in it.
• [Jan 06,2026] 🚀 Code Release: Training code and scripts are now available! Please ⭐ Star this repo to stay tuned!

Supervised Fine-Tuning (SFT) is the standard paradigm for domain adaptation, yet it frequently incurs the cost of 🎯 catastrophic forgetting. In sharp contrast, on-policy Reinforcement Learning (RL) effectively preserves general capabilities.

We investigate this discrepancy and identify a fundamental distributional gap: while RL aligns with the model's internal belief, SFT forces the model to fit external supervision. This mismatch often manifests as "💥 Confident Conflicts"—tokens characterized by low probability but low entropy. In these instances, the model is highly confident in its own prediction but is forced to learn a divergent ground truth, triggering destructive gradient updates.

To address this, we propose 🚀 Entropy-Adaptive Fine-Tuning (EAFT). Unlike methods relying solely on prediction probability, EAFT utilizes token-level entropy as a gating mechanism to distinguish between epistemic uncertainty and knowledge conflict. This allows the model to learn from uncertain samples while suppressing gradients on conflicting data.

🔬 Key Findings:

• Extensive experiments on Qwen and GLM series (4B-32B parameters) across mathematical, medical, and agentic domains
• EAFT consistently matches the downstream performance of standard SFT
• Significantly mitigates the degradation of general capabilities
• "The right balance between learning and forgetting"

Figure 1: (a) Conceptual illustration of Confident Conflicts. (b) Token-level entropy-probability landscape comparison between SFT and On-Policy Rollouts.

We identify "Confident Conflicts" (Low Probability, Low Entropy) as the primary driver of catastrophic forgetting. These occur when:

• The model is confident about its prediction (low entropy)
• But the prediction conflicts with ground truth (low probability)

EAFT introduces an entropy-based gating mechanism to the standard Cross-Entropy loss:

$$\mathcal{L}_{EAFT} (\theta) = - \sum_{t=1}^{T} \tilde{H}_t \cdot \log P_\theta(y_t | x, y_{<t})$$

Where $\tilde{H}_t$ is the normalized entropy. This mechanism:

1. ⚠️ Suppresses Conflicts: Down-weights gradients when the model is stubborn (Low Entropy), preventing destructive updates
2. ✨ Encourages Learning: Maintains high weights when the model is uncertain/exploring (High Entropy)

Unlike methods that rely solely on prediction probability, EAFT utilizes token-level entropy to distinguish between:

• Epistemic uncertainty: Model doesn't know → Strong learning signal
• Knowledge conflict: Model is confident but wrong → Suppressed updates

Visualization: EAFT effectively suppresses destructive gradients in the Confident Conflict region.

🩺 Medical Domain (Base: Qwen3-4B-Thinking)

🤖 Agent Domain (Base: Qwen3-4B-Instruct)

• Step 1: Clone the repository

git clone https://github.com/ymxyll/LlamaFactory-EAFT.git
cd LlamaFactory-EAFT

• Step 2: Install dependencies

pip install -e .

• Step 3: Run the training script

llamafactory-cli train --config examples/extras/eaft/qwen25_05b_eaft_full.yaml

• Step 1: Clone the repository

git clone https://github.com/ymxyll/ms-swift-EAFT.git
cd ms-swift-EAFT

• Step 2: Install dependencies

pip install -e .

• Step 3: Run the training script

# megatron
bash examples/megatron/eaft.sh
# deepspeed
bash examples/train/eaft.sh

If you find this work helpful for your research, please consider citing our paper:

@article{diao2026entropy,
  title={Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting},
  author={Diao, Muxi and Yang, Lele and Gong, Wuxuan and Zhang, Yutong and Yan, Zhonghao and Han, Yufei and Liang, Kongming and Xu, Weiran and Ma, Zhanyu},
  journal={arXiv preprint arXiv:2601.02151},
  year={2026}
}