GPUs vs CPUs

In order to run the millions of calculations at once required for LLMs, the AI computers in the data centres use a special computer chip, or processor, called a GPU.

Unlike the CPU (or brain) in your laptop, a GPU is designed to split a big problem into many smaller parts and work on them at the same time.

GPU stands for Graphics Processing Unit because they were first designed to process images and graphics. You have a smaller version of a GPU on your laptop.

Let's look at a desktop computer with the case open to see a GPU and CPU. The picture below shows a desktop computer with the GPU and CPU labelled. The CPU is underneath a fan but if we take the fan off and look closer (in the right part of the picture), the CPU is the smaller silver square shown in the image.

CPUs are better for jobs when the computer has to make lots of different decisions in order, while GPUs are better when the same type of calculation needs to happen many times at once.

Best for CPU:

• Simple coding tasks: can follow instructions in order
• Running a spreadsheet with formulas: the formulas usually depend on each other

Best for GPU:

• Video editing: can process multiple frames at once
• 3D animation: can render many scenes in parallel
• Training and Running LLMs: the transformer layers can happen all at the same time