Skip to content
Back to System DesignAll lessons

System Design

Content-Addressable Storage (Address by Hash)

O(n) time to hash content of length n · O(1) time to store or look up once you already have the hash · space grows with the number of distinct contents stored, not the number of times something was saved.

The idea, in plain English

Imagine a library where books aren't filed by title. Instead, each book is filed by a fingerprint computed from its exact contents — the fingerprint is the shelf location. If two people bring in the exact same book, with identical content, it lands on the exact same shelf spot. So the library only ever keeps one copy, no matter how many times someone 'donates' it. Change even one word, and the fingerprint changes completely. It lands on a brand-new shelf spot, as a different book. That's content-addressable storage. Instead of storing data under a name someone chose, you store it at an address computed from the data's own content. Identical content always lands in the same place — automatic deduplication. Any change gets a brand-new address.

How it works

  1. 1To store a piece of data, compute a hash of its exact content. The same content always produces the same hash. This hash becomes the data's address.
  2. 2Save the data in a lookup table, keyed by that hash. If something with that exact hash is already stored, there's nothing new to do — it's already there. That's deduplication for free.
  3. 3To retrieve data, you just need its hash — its address. Look it up directly; you need no separate naming system. Any change to the content produces a different hash. So old content can never be silently overwritten — you simply end up with two different addresses.

When you'd use it

Use content-addressable storage once your app is popular and you're storing lots of files or blobs, many of which turn out to be byte-for-byte identical. Git storing file contents, a Docker image storing layers, and a backup system storing chunks are good examples. This approach automatically avoids storing the same content twice. It also lets you check that data hasn't been tampered with, by recomputing the hash and comparing it.

Common beginner mistakes

  • Using a hash too weak, so two different pieces of content can produce the same address — a 'collision.' Real systems use cryptographic hashes to make this astronomically unlikely. This toy example uses a simple sum-of-character-codes hash just to stay simple and predictable.
  • Assuming you can 'update' the content stored at a given address. You can't — changing the content changes its hash entirely. Content-addressable storage never changes existing entries. An 'update' is really just storing new content at a new address.

Try it — edit and run

Click the code to edit · press ⌘/Ctrl+↵ to run

Editable code. Tab and Shift+Tab indent. Press Escape, then Tab, to move focus out of the editor.

Expected output — hit Run to try it
Storing content-addressed blobs:
"hello world" -> addr-1116 (stored)
"goodbye world" -> addr-1329 (stored)
"hello world" -> addr-1116 (already stored (dedup))
"hello world!" -> addr-1149 (stored)
Distinct blobs stored: 3 out of 4 put() calls
get(addr-1116) -> "hello world"

Not sure this is the right topic? See the learning paths → or where this leads →