preference tuning

Criterion RLHF (reward model + PPO) DPO Planning use Reward model Built from preference data first No explicit reward function;…