# Cache-Aside

> Read through a cache: on a miss, load from the store and fill it; on a write, save to the store and evict the entry - so reads speed up while the store stays the only source of truth.

- kind: pattern
- category: data
- aliases: Lazy Loading, Look-Aside Cache
- canonical: https://aipatterns.dev/design-patterns/cache-aside
- inspired-by: https://learn.microsoft.com/en-us/azure/architecture/patterns/cache-aside

## Intent

Keep a cache beside the store of record, not inside it. On a read, check the cache first; on a miss, load from the store, put the value in the cache, and return it. On a write, save to the store and drop the cache entry, so the next read reloads the fresh value. The cache is an accelerator that can be emptied at any time; the store remains the only source of truth.

## Forces

- **Read speed** vs **write cost**: every cached read saves a query; every write now has a second thing to keep honest.
- **Staleness** vs **consistency**: a cached answer can be out of date until its entry expires or is evicted.
- **Eager filling** vs **wasted work**: pre-warming the cache with everything costs memory and effort for values that may never be read again.
- **Two copies** vs **one truth**: once the cache holds data, someone will eventually ask which copy is right.

## Therefore

**On a miss, load from the store and fill the cache; on a write, update the store and evict - never update both and hope they stay in step.**

The application code owns the coordination: check the cache, fall back to the store, fill on the way out. The cache is disposable by construction - empty it and the system is slower, never wrong. Writes go to the store alone and invalidate the entry, because the alternative (writing to both) turns every failure between the two into silent disagreement.

## Structure

```mermaid
classDiagram
  class PricingService {
    -Cache cache
    -PricingStore store
    +priceFor(sku) Price
    +reprice(sku, cents)
  }
  class Cache {
    +get(key) Price
    +put(key, price)
    +evict(key)
  }
  class PricingStore {
    +load(sku) Price
    +save(sku, cents)
  }
  PricingService --> Cache : reads, fills, evicts
  PricingService --> PricingStore : loads on miss, saves on write
```

### Participants

- **Coordinator** (`PricingService`): the application code the pattern lives in. Implements the read path (cache, then store, then fill) and the write path (store, then evict).
- **Cache** (`Cache`): a keyed, expirable store - Redis, Memcached, or an in-process map. Holds entries with a TTL; losing everything is acceptable.
- **Store** (`PricingStore`): the source of truth - the database. Every fact in the cache can be regenerated from it.
- **Clock** (`now`): injected so TTL expiry is testable without sleeping.

## Use it when

- Reads dominate writes on data that is expensive to compute or far away: product pages, user profiles, configuration, an LLM response keyed by prompt hash.
- Slightly stale answers are acceptable within a bounded window - a price that is at most sixty seconds old.
- The working set is bigger than the cache, so pre-warming everything would waste memory; entries should earn their place by being read.
- You already have a cache (Redis, a CDN, an in-process map) and a store, and need the rule that keeps them from drifting apart.

## Do not use it when

- The read must reflect a write made moments ago in the same session - showing a user their just-saved settings from a stale cache is a bug. Bypass the cache for that read, or key the cache per session.
- The value must always be current for correctness - stock levels at checkout, an account balance before a debit. Read the store; cache only immutable or slowly changing derivatives.
- Writes are as frequent as reads. Every write pays an eviction and the hit rate never covers it; write to the store alone.
- The underlying query is already fast, measured. A cache adds a second source of truth and an invalidation surface to save milliseconds you have not proven you need.
- You are in one process with the data already in memory. A map beside your map is not a cache layer, it is a bug incubator; reach for this when the store is remote or the computation is expensive.

## Application grill

**Inspect first:** the read paths the pattern would accelerate and what they cost today (a database hit per request, N+1 loops, remote calls); whether a cache already exists - search for `redis`, `ioredis`, `memcached`, `lru-cache`, `node-cache`, framework caches (`unstable_cache`), `Cache-Control` headers, and in-process maps near the store; the deploy target and instance count, which decide whether one per-process cache is enough.

**Missing pieces:** with no cache in the codebase, the pattern needs one. An in-process LRU (a size-capped map) is free to run but per-instance: two instances cache twice and invalidate independently. Managed Redis (Upstash, ElastiCache, Memorystore, or the platform's Redis/KV binding) is shared across instances and adds a service to pay for, secure and monitor. A CDN may already cache whole responses for anonymous traffic - check before adding an application cache.

**Ask before you build:**

- **Scale:** how many reads per second land on the hot path, and how large is the working set that would live in the cache?
- **Performance:** what latency is being bought, and how stale may an answer be (the TTL)?
- **Reliability:** is "slower, never wrong" the agreed failure mode when the cache is down, and can the store survive a hot-key stampede on expiry?
- **Budget:** pay for a managed cache now, or start in-process and accept per-instance scope?
- **Implementation:** who owns invalidation on writes, and which single read path becomes the coordinator?

## Implementation steps

1. Name the cache key precisely: the query and its parameters (`price:{sku}`), not the entity (`product:{id}`), when different queries need different shapes.
2. Write the read path in one place: get from cache; on miss, load from the store, put with a TTL, return. Every reader goes through it - a second read path is an uncacheable read path.
3. Write the write path in the same place: save to the store, then evict the key. Evict, do not overwrite - the store's write is the fact; the cache entry is only a speedup.
4. Give every entry a TTL even though writes evict, so bugs, rollbacks and manual edits age out instead of living forever.
5. Decide the miss storm policy before you need it: when a hot key expires, many readers miss at once. Fill under a per-key lock, or accept the thundering herd because the store can take it.
6. Measure the hit rate. A cache below roughly half hits is cost without payoff; reconsider the key or the TTL.
7. Log evictions and misses at the key level while integrating, then drop to metrics - the first production bug in every cache is a wrong key.

## Example

A storefront reads prices on every page view and reprices a few times a day. The first read misses and loads; the second is served from cache; a reprice saves to the store and evicts, so the next read reloads the fresh value; after the TTL expires, even an unrepriced entry reloads.

```ts title="pricing.ts"
type Price = { cents: number };

interface PricingStore {
  load(sku: string): Price;
  save(sku: string, cents: number): void;
}

/** The store of record: counts loads so the demo can prove caching happened. */
class SlowDatabase implements PricingStore {
  loads = 0;
  private prices = new Map<string, number>([["tea-1", 450]]);

  load(sku: string): Price {
    this.loads++;
    return { cents: this.prices.get(sku) ?? 0 };
  }
  save(sku: string, cents: number): void {
    this.prices.set(sku, cents);
  }
}

class Cache {
  private entries = new Map<string, { value: Price; at: number }>();
  constructor(
    private readonly ttlMs: number,
    private readonly now: () => number,
  ) {}

  get(key: string): Price | undefined {
    const e = this.entries.get(key);
    if (!e) return undefined;
    if (this.now() - e.at >= this.ttlMs) {
      this.entries.delete(key); // expired: a miss, not a stale answer
      return undefined;
    }
    return e.value;
  }
  put(key: string, value: Price): void {
    this.entries.set(key, { value, at: this.now() });
  }
  evict(key: string): void {
    this.entries.delete(key);
  }
}

class PricingService {
  constructor(
    private readonly store: PricingStore,
    private readonly cache: Cache,
  ) {}

  priceFor(sku: string): Price {
    const key = `price:${sku}`;
    const hit = this.cache.get(key);
    if (hit) return hit; // fast path
    const price = this.store.load(sku); // miss: the store is the truth
    this.cache.put(key, price); // fill on the way out
    return price;
  }

  reprice(sku: string, cents: number): void {
    this.store.save(sku, cents); // write the fact
    this.cache.evict(`price:${sku}`); // drop the stale copy, never overwrite it
  }
}

const store = new SlowDatabase();
let clock = 0;
const service = new PricingService(store, new Cache(1_000, () => clock));

service.priceFor("tea-1"); // miss -> load 1
service.priceFor("tea-1"); // hit
console.log(`loads after two reads: ${store.loads}`); // 1

service.reprice("tea-1", 475); // save + evict
console.log(service.priceFor("tea-1").cents); // 475, load 2: fresh, not overwritten

clock += 2_000; // TTL (1 s) has passed
service.priceFor("tea-1"); // expired -> load 3
console.log(`loads after expiry: ${store.loads}`); // 3
```

```python title="pricing.py"
from dataclasses import dataclass
from typing import Callable, Protocol

@dataclass
class Price:
    cents: int

class PricingStore(Protocol):
    def load(self, sku: str) -> Price: ...
    def save(self, sku: str, cents: int) -> None: ...

class SlowDatabase:
    """The store of record: counts loads so the demo can prove caching happened."""

    def __init__(self) -> None:
        self.loads = 0
        self._prices = {"tea-1": 450}

    def load(self, sku: str) -> Price:
        self.loads += 1
        return Price(cents=self._prices.get(sku, 0))

    def save(self, sku: str, cents: int) -> None:
        self._prices[sku] = cents

class Cache:
    def __init__(self, ttl_ms: int, now: Callable[[], int]) -> None:
        self._ttl, self._now = ttl_ms, now
        self._entries: dict[str, tuple[Price, int]] = {}

    def get(self, key: str) -> Price | None:
        entry = self._entries.get(key)
        if entry is None:
            return None
        price, at = entry
        if self._now() - at >= self._ttl:
            del self._entries[key]  # expired: a miss, not a stale answer
            return None
        return price

    def put(self, key: str, value: Price) -> None:
        self._entries[key] = (value, self._now())

    def evict(self, key: str) -> None:
        self._entries.pop(key, None)

class PricingService:
    def __init__(self, store: PricingStore, cache: Cache) -> None:
        self._store, self._cache = store, cache

    def price_for(self, sku: str) -> Price:
        key = f"price:{sku}"
        hit = self._cache.get(key)
        if hit is not None:
            return hit  # fast path
        price = self._store.load(sku)  # miss: the store is the truth
        self._cache.put(key, price)  # fill on the way out
        return price

    def reprice(self, sku: str, cents: int) -> None:
        self._store.save(sku, cents)  # write the fact
        self._cache.evict(f"price:{sku}")  # drop the stale copy, never overwrite it

store = SlowDatabase()
clock = 0
now = lambda: clock  # noqa: E731 - injected clock; reassigning moves time for both
service = PricingService(store, Cache(ttl_ms=1_000, now=now))

service.price_for("tea-1")  # miss -> load 1
service.price_for("tea-1")  # hit
print(f"loads after two reads: {store.loads}")  # 1

service.reprice("tea-1", 475)  # save + evict
print(service.price_for("tea-1").cents)  # 475, load 2: fresh, not overwritten

clock += 2_000  # TTL (1 s) has passed
service.price_for("tea-1")  # expired -> load 3
print(f"loads after expiry: {store.loads}")  # 3
```

```go title="pricing.go"
package main

import "fmt"

type Price struct{ Cents int }

// PricingStore is the store of record.
type PricingStore interface {
	Load(sku string) Price
	Save(sku string, cents int)
}

// SlowDatabase counts loads so the demo can prove caching happened.
type SlowDatabase struct {
	loads  int
	prices map[string]int
}

func NewSlowDatabase() *SlowDatabase {
	return &SlowDatabase{prices: map[string]int{"tea-1": 450}}
}

func (d *SlowDatabase) Load(sku string) Price {
	d.loads++
	return Price{Cents: d.prices[sku]}
}

func (d *SlowDatabase) Save(sku string, cents int) {
	d.prices[sku] = cents
}

type cacheEntry struct {
	value Price
	at    int
}

type Cache struct {
	ttlMs   int
	now     func() int
	entries map[string]cacheEntry
}

func NewCache(ttlMs int, now func() int) *Cache {
	return &Cache{ttlMs: ttlMs, now: now, entries: map[string]cacheEntry{}}
}

func (c *Cache) Get(key string) (Price, bool) {
	e, ok := c.entries[key]
	if !ok {
		return Price{}, false
	}
	if c.now()-e.at >= c.ttlMs {
		delete(c.entries, key) // expired: a miss, not a stale answer
		return Price{}, false
	}
	return e.value, true
}

func (c *Cache) Put(key string, value Price) {
	c.entries[key] = cacheEntry{value: value, at: c.now()}
}

func (c *Cache) Evict(key string) {
	delete(c.entries, key)
}

type PricingService struct {
	store PricingStore
	cache *Cache
}

func NewPricingService(store PricingStore, cache *Cache) *PricingService {
	return &PricingService{store: store, cache: cache}
}

func (s *PricingService) PriceFor(sku string) Price {
	key := "price:" + sku
	if hit, ok := s.cache.Get(key); ok {
		return hit // fast path
	}
	price := s.store.Load(sku) // miss: the store is the truth
	s.cache.Put(key, price)    // fill on the way out
	return price
}

func (s *PricingService) Reprice(sku string, cents int) {
	s.store.Save(sku, cents)          // write the fact
	s.cache.Evict("price:" + sku)     // drop the stale copy, never overwrite it
}

func main() {
	store := NewSlowDatabase()
	clock := 0
	now := func() int { return clock }
	service := NewPricingService(store, NewCache(1_000, now))

	service.PriceFor("tea-1") // miss -> load 1
	service.PriceFor("tea-1") // hit
	fmt.Printf("loads after two reads: %d\n", store.loads) // 1

	service.Reprice("tea-1", 475)  // save + evict
	fmt.Println(service.PriceFor("tea-1").Cents) // 475, load 2: fresh, not overwritten

	clock += 2_000 // TTL (1 s) has passed
	service.PriceFor("tea-1") // expired -> load 3
	fmt.Printf("loads after expiry: %d\n", store.loads) // 3
}
```

```java title="Pricing.java"
import java.util.HashMap;
import java.util.Map;
import java.util.function.IntSupplier;

public class Pricing {
    record Price(int cents) {}

    interface PricingStore {
        Price load(String sku);

        void save(String sku, int cents);
    }

    /** The store of record: counts loads so the demo can prove caching happened. */
    static final class SlowDatabase implements PricingStore {
        int loads = 0;
        private final Map<String, Integer> prices = new HashMap<>(Map.of("tea-1", 450));

        public Price load(String sku) {
            loads++;
            return new Price(prices.getOrDefault(sku, 0));
        }

        public void save(String sku, int cents) {
            prices.put(sku, cents);
        }
    }

    static final class Cache {
        private record Entry(Price value, long at) {}

        private final long ttlMs;
        private final IntSupplier now;
        private final Map<String, Entry> entries = new HashMap<>();

        Cache(long ttlMs, IntSupplier now) {
            this.ttlMs = ttlMs;
            this.now = now;
        }

        Price get(String key) {
            Entry e = entries.get(key);
            if (e == null) return null;
            if (now.getAsLong() - e.at() >= ttlMs) {
                entries.remove(key); // expired: a miss, not a stale answer
                return null;
            }
            return e.value();
        }

        void put(String key, Price value) {
            entries.put(key, new Entry(value, now.getAsLong()));
        }

        void evict(String key) {
            entries.remove(key);
        }
    }

    static final class PricingService {
        private final PricingStore store;
        private final Cache cache;

        PricingService(PricingStore store, Cache cache) {
            this.store = store;
            this.cache = cache;
        }

        Price priceFor(String sku) {
            String key = "price:" + sku;
            Price hit = cache.get(key);
            if (hit != null) return hit; // fast path
            Price price = store.load(sku); // miss: the store is the truth
            cache.put(key, price); // fill on the way out
            return price;
        }

        void reprice(String sku, int cents) {
            store.save(sku, cents); // write the fact
            cache.evict("price:" + sku); // drop the stale copy, never overwrite it
        }
    }

    public static void main(String[] args) {
        SlowDatabase store = new SlowDatabase();
        long[] clock = {0};
        PricingService service =
                new PricingService(store, new Cache(1_000, () -> clock[0]));

        service.priceFor("tea-1"); // miss -> load 1
        service.priceFor("tea-1"); // hit
        System.out.println("loads after two reads: " + store.loads); // 1

        service.reprice("tea-1", 475); // save + evict
        System.out.println(service.priceFor("tea-1").cents()); // 475, load 2: fresh

        clock[0] += 2_000; // TTL (1 s) has passed
        service.priceFor("tea-1"); // expired -> load 3
        System.out.println("loads after expiry: " + store.loads); // 3
    }
}
```

Under concurrency the same rule holds with one addition: the write path saves and evicts as one logical step, and a read that filled the cache before the eviction landed may briefly serve the old value - that is the staleness window the TTL bounds, not a bug to chase.

## Trade-offs

**Gains**
- Reads stop paying the store's cost; the cache absorbs the hot path.
- The store stays the only source of truth - empty the cache and nothing is wrong, just slow.
- Failures degrade in the safe direction: a cache outage means slower reads, never wrong data.

**Costs**
- Answers are stale by up to the TTL; read-your-writes needs a bypass or per-session keys.
- Every write now has two steps to keep ordered, and the invalidation is the part everyone forgets.
- Hot-key expiry can stampede the store; per-key fill locking adds real complexity.
- A second datastore to operate, monitor and evict.

## Relations

- [Proxy](https://aipatterns.dev/design-patterns/proxy.md): the coordinator exposes the store's read interface and adds caching in front - the same shape, but the pattern's point is the invalidation rule, not the wrapping.
- [Fallback](https://aipatterns.dev/design-patterns/fallback.md): a stale cached answer served during a store outage is the most useful fallback there is - the cache you already have doubles as the degraded mode.
- [Strategy](https://aipatterns.dev/design-patterns/strategy.md): eviction and TTL policies vary (LRU, LFU, per-key TTL); swap them without touching the read path.
- [Flyweight](https://aipatterns.dev/design-patterns/flyweight.md): both avoid repeating expensive work by sharing, but flyweight shares objects inside one process to save memory, while cache-aside shares computed answers across processes to save loads.

## Agent checklist

- [ ] Reads and writes go through one coordinator each; no second read path skips the cache.
- [ ] Writes save to the store and evict; nothing overwrites a cache entry directly.
- [ ] Every entry has a TTL, and expiry is a miss rather than a stale serve.
- [ ] The cache key covers everything the value depends on.
- [ ] The hot-key expiry path is decided (lock or accept), and the hit rate is measured.