Reinforcement Learning from Human Feedback

Reinforcement Learning from Human Feedback (RLHF) is a training approach where real humans rank responses to certain prompts, and these rankings are 'fed back' into the Transformer to train it to improve the model output.

RLHF is a training process applied to the whole model after it has already been built and pre-trained.

OpenAI's breakthrough with the first ChatGPT in November 2022 was fine-tuning their massive pre-trained GPT model using real human feedback to train it to be more helpful and conversational like a real assistant.