Embedding Arithmetic

"Man is to woman as king is to ___?" Most people instantly answer queen. Embeddings, the number-list representation of a word's meaning (see Cosine Similarity), can play this exact game using plain arithmetic. Take king's numbers, subtract man's numbers, then add woman's numbers. The result vector lands almost exactly where queen's vector already sits. This is not a coincidence. It means the "shift" from man to woman is baked into the embedding space as a consistent direction. Adding that same shift to king slides you over to queen.

1. 1Start with three toy embeddings, which are small lists of numbers, for king, man, and woman.
2. 2Do the arithmetic position by position. Subtract man's vector from king's vector, which removes 'maleness'. Then add woman's vector, which adds 'femaleness' back in. This gives you a brand-new result vector.
3. 3A real system does not know in advance which word that result vector 'means'. So you search a small vocabulary of candidate word embeddings, and use cosine similarity (see Cosine Similarity) to find whichever one the result vector is closest to.

This is a neat demonstration that embedding spaces encode relationships, such as gender, tense, or country-capital pairs, as consistent directions, not just individual word meanings. It is the classic proof-of-concept behind early embedding models like word2vec.

• ⚠Don't assume this works perfectly for every word pair in real embeddings. The king, man, woman example is famous specifically because it works cleanly. Many analogies come out messier in practice.
• ⚠Remember that the arithmetic alone does not produce a word. The result is just another point in the vector space. You still need a similarity search over real words to turn it back into something readable.