Text Prediction

You've probably noticed your phone trying to finish your sentences. This technology is called text prediction, which shares some similarities with how LLMs generate text.

In this page, we explain how text prediction with small models can work, with an example for you to try. But first, read through the sections below to learn how text prediction on your phone works.

Text prediction example

You can generate a sentence using an example text prediction model by clicking the "Next word" button in the interactive below.

The model uses bigrams to predict the next word and is trained on sentences like "he thought he ate that up for real" and "for real bro like how do you recover". The training data and how the model chooses words is explained in more detail in the next section.

How the model works

The model uses bigrams to choose the next word in the generated sentence. This means it looks at the current word and chooses the next word at random from the words that appear after the current word in the training data.

For example, let's focus on these two sentences:

• for real bro it was like so embarrassing
• for real bro like how do you recover

If we look at the word "bro" in the first sentence, the word that appears after it is "it". In the second sentence, the word that appears after "bro" is "like".

Let's say we only used these sentences for the text prediction model. Then if the model was asked to predict the next word after "bro", it would choose between "it" and "like" at random.

If we created a table of possible words that could come after "bro", like the tables in the interactive above, it would look like this:

On the other hand, if we look at the word "real", the word that appears after it is "bro" in the two sentences. The table with possible words would look like this:

In that case, the model would just choose "bro" as the next word every time as it is the only possible word that follows "real".

Most models like this will be choose the next word at random. However, the probability of choosing each word is based on how common it is in the training data. In the interactive above, you will see this when the current word is "for". The word "real" appears 5 times after "for" and the word "reals" appears once. This means that the model will choose "real" as the next word 5 out of 6 times.

The text prediction model we used for the interactive above was trained on more than two sentences. You can click the button below to see the training data used.

The training data used for text prediction models on your phone and other machines would use a lot more data than this (thousands of sentences, rather than just 6).