Predicting the next token

The model then predicts possible next tokens by giving every possible token (~50000) in its vocabulary a score. For example in:

the probabilities of the next token might look like this:

with the remaining ~49995 tokens having very low probabilities. And all these probabilities add up to 1.

The model then usually picks the token with the highest probability, adds it to the end of the sentence and repeats the process until it reaches a stopping point.