Softmax & Temperature

A language model does not just blurt out one 'next word'. First, it gives every candidate word a raw score, called a logit, for how likely it seems. Softmax is the step that turns those raw scores into real percentages that add up to 100%. Think of it like a poll of the model's confidence in each word. Temperature is a dial you turn before that happens. Turn it down, say to 0.5, and the model gets more confident and predictable, almost always picking the top word. Turn it up, say to 2.0, and the scores flatten out. This gives less-likely words a real shot at being picked, making the text more surprising and more 'creative'.

1. 1Start with raw scores, called 'logits', for each candidate. There is one number per option, and it can be any size.
2. 2First, divide every score by the temperature. A smaller temperature makes the biggest scores stand out even more, giving a sharper result. A larger temperature squashes the differences, giving a flatter result.
3. 3Apply softmax. Raise e to the power of each scaled score, then divide each result by the total of all of them. This turns the scores into probabilities that add up to 1.

Use this to decide how 'creative' or how 'predictable' text generation should be. Choose a low temperature for factual, consistent answers, and a higher temperature for brainstorming or varied writing.

• ⚠Don't set temperature to 0 and divide directly. That causes a division by zero. Real systems instead treat temperature 0 as a special case that always picks the top score.
• ⚠Remember that probabilities are rounded for display. Rounding each one to 2 decimal places on its own can make them add up to 1.01 or 0.99 instead of exactly 1.00. This is normal rounding, not a bug.