68 Caching Interview Questions and Answers (2026)

A cache is a fast, usually small store that keeps copies of data so future requests are served quickly without recomputing or re-fetching from the slower source of truth.

• Reduce latency: Serve from fast memory instead of a slow disk, database, or remote API.

• Reduce load on the backend: Absorb repeated reads so the origin database/service handles fewer queries and scales further.

• Increase throughput: Cheaper per-request work means the same hardware serves more requests per second.

• Save expensive computation: Store results of costly operations (aggregations, rendered pages, ML features) instead of recomputing.

• Where caches live: At many layers: CPU caches, in-process memory, distributed stores like Redis, CDNs, and browsers.

• The core trade-off: Speed in exchange for possible staleness and the complexity of keeping the cache consistent with the source.

A hit is when requested data is found in the cache; a miss is when it isn't and must be fetched from the source. 'Cold' vs 'warm' describes how populated the cache is: a cold cache misses a lot, a warm one is full of useful entries and hits often.

• Cache hit: Data is present and valid, so it is returned fast with no origin call.

• Cache miss:

  • Data is absent (or expired), so the system fetches from the source, returns it, and usually populates the cache for next time.

  • Misses cost the lookup plus the origin fetch, so a high miss rate erodes the cache's value.

• Hit rate: hits / (hits + misses): the headline metric for whether a cache is earning its keep.

• Cold cache: Just started/flushed, mostly empty, so early requests mostly miss (these are 'compulsory' or cold-start misses).

• Warm cache:

  • Already holds the hot working set, so it hits frequently and delivers its intended speedup.

  • Warming (prefetching popular keys on startup) avoids a cold-start latency spike after a deploy or restart.

Memoization is a specific form of caching: it stores the result of a pure function keyed by its arguments. Caching is the broader concept of storing any expensive-to-produce data (DB rows, HTTP responses, rendered pages) for reuse, often across processes and machines.

• Memoization:

  • Scope: a single function's return value, keyed by its inputs.

  • Assumes purity: same args produce same result, so it's safe to reuse.

  • Usually in-process and in-memory (e.g. functools.lru_cache).

• Caching:

  • Scope: any data, from any source, at any layer (browser, CDN, app, DB).

  • Often distributed (Redis, Memcached) and shared across instances.

  • Must handle invalidation, TTLs, and staleness because the source can change independently.

• Relationship: all memoization is caching, but not all caching is memoization.

When the cache is full and no TTLs have expired, a new write triggers the eviction policy: the cache makes room by removing an existing (still-valid) entry according to its configured policy, or it rejects the write if configured to. TTL and capacity are independent constraints.

• Eviction kicks in: A policy (LRU, LFU, random, etc.) picks a victim to delete even though it hasn't expired, so the new entry fits.

• TTL vs. eviction are different: TTL is time-based expiry of correctness/freshness; eviction is space-based reclamation under memory pressure. Either can remove an entry.

• Policy depends on configuration:

  • Redis, for example, supports modes like `allkeys-lru`, `volatile-lru` (only evict keys that have a TTL set), `allkeys-lfu`, and `noeviction`.

  • Under `noeviction`, the cache refuses new writes and returns an error instead of removing anything.

• Implication: Long TTLs don't guarantee an item survives; a hot keyspace larger than memory means valid entries get evicted early, lowering hit ratio.

A 304 Not Modified means the cached copy is still valid, so the body is not re-sent: the client reuses its stored representation and the cache's freshness is refreshed using the new response headers.

• No body transfer: The 304 has empty body, saving bandwidth; the client serves the existing cached payload.

• Freshness is reset: Headers on the 304 (updated Cache-Control, Expires, Date) update the stored entry, restarting its TTL.

• The entry becomes usable again: What was stale (and triggered the conditional GET) is now treated as fresh until it next expires.

A fixed TTL expires an entry a set time after it was written, regardless of usage; a sliding TTL resets the expiration timer on each access, so frequently-used data stays cached and idle data ages out.

• Fixed (absolute) TTL:

  • Lifetime is bounded from write time: "expire 60s after creation" no matter how often it's read.

  • Guarantees a maximum staleness, which is good for correctness-sensitive data.

• Sliding (rolling) TTL:

  • Each read pushes expiry forward: "expire 60s after last access."

  • Keeps hot data resident and naturally evicts cold data, ideal for session tokens.

• Key risk of sliding TTL: A continuously-accessed entry may never expire, so it can serve indefinitely stale data; often paired with an absolute cap (max lifetime).

An in-process cache lives inside your application's own memory (same heap as the app), so access is extremely fast but local to that one instance. A distributed cache is a separate service (e.g. Redis, Memcached) shared over the network by all instances, trading some latency for a single consistent shared view and far larger capacity.

• In-process (local):

  • Nanosecond/microsecond access, no serialization or network hop.

  • Limited by the process's RAM; each instance has its own copy, so data can be inconsistent across nodes and is lost on restart.

  • Examples: Caffeine, Guava, a simple in-memory dict.

• Distributed (remote):

  • Shared across all app instances, so reads/writes are consistent and capacity scales independently.

  • Costs a network round trip plus serialization; the cache itself becomes infrastructure to operate and a potential failure point.

• When to choose which:

  • Local: small, hot, mostly-read, tolerant-of-staleness data where lowest latency matters.

  • Distributed: data that must be consistent across instances, too large to replicate per node, or must survive instance restarts.

  • Common pattern: a near-cache combining both (local L1 in front of a distributed L2).

Caching is a bad idea when the cost of staleness, complexity, or low hit rates outweighs the latency you save: it adds a moving part that can serve wrong data or fail in surprising ways.

• Low hit rate / poor locality: If access is uniformly random or each key is read once, you pay the cache lookup and population cost but rarely benefit.

• Strong consistency requirements: Financial balances, inventory counts, auth state: serving stale data is a correctness bug, not a slowdown.

• Highly volatile data: If data changes faster than it is read, you invalidate before you reuse, so the cache is pure overhead.

• Reliability risks it introduces:

  • A cache stampede (thundering herd) on expiry can hammer the origin worse than no cache.

  • A cache outage can cause a load spike the origin was never sized for, turning an optimization into a single point of failure.

• Small or cheap data: If the source is already fast (in-memory, local), the network hop to a remote cache can be slower than recomputing.

Locality says that recent and nearby accesses predict future accesses, so a small fast store can satisfy most requests: this is exactly the bet a cache makes.

• Temporal locality:

  • If an item was accessed recently, it is likely to be accessed again soon, so keeping it close pays off repeatedly.

  • Justifies eviction policies like LRU that keep recently-used items.

• Spatial locality:

  • If an item is accessed, nearby items are likely next, so fetching in blocks/pages amortizes the cost.

  • Justifies prefetching and reading larger chunks than strictly requested.

• Why it works economically: Real workloads are skewed (a small hot set gets most traffic), so a cache far smaller than the full dataset still serves a high fraction of requests.

• The flip side: With no locality (random scans over a huge keyspace), the assumption breaks and the cache provides little value.

Frequency alone doesn't justify caching: if the data must be perfectly fresh, is expensive to store safely, or is cheap to fetch anyway, caching can hurt more than help.

• Correctness sensitivity: Data where staleness causes harm (account balance, permissions, real-time price) should be read from source even if hot.

• High churn: If it is written almost as often as read, entries are invalidated before reuse, wasting cache space and invalidation effort.

• Cheap or already-fast source: If the origin lookup is faster than a remote cache round trip, caching adds latency.

• Security / privacy: Sensitive per-user data in a shared cache risks leakage and broadens the attack surface.

• Large payload, low value: A huge object can evict many useful small entries, hurting overall hit rate (poor cost-per-byte).

A cache adds a lookup before the source, so when hits are rare or the cache is slower/farther than expected, you pay extra work on most requests instead of saving time.

• The miss penalty: Every miss costs cache-lookup time PLUS the origin fetch PLUS the write-back: with a low hit rate this net-adds latency.

• Network hop to a remote cache: A round trip to Redis/Memcached can exceed an in-process or already-fast DB lookup, especially for tiny values.

• Serialization overhead: Marshalling/unmarshalling large objects to store and retrieve can dominate the time saved.

• Write-path amplification: Write-through or aggressive invalidation makes writes slower because they touch both store and cache.

• Stampedes and contention: On popular-key expiry, many clients miss at once and queue behind the origin, spiking tail latency.

A good cache key uniquely and deterministically identifies the cached value, is stable across requests for the same logical input, and includes every input that affects the result. It should be compact, readable, and collision-free.

• Deterministic and complete: Encode all inputs that change the output (user, locale, version, query params). Missing an input returns the wrong cached value to someone.

• Namespaced and versioned: Prefix by domain and schema version (e.g. user:v2:123:profile) so you can invalidate or migrate a whole group at once.

• Canonical / normalized: Normalize order and casing so ?a=1&b=2 and ?b=2&a=1 map to the same key.

• Pitfalls to avoid:

  • Over-specific keys: including volatile fields (timestamps, request IDs) destroys hit rate.

  • Under-specific keys: omitting an input (e.g. locale) causes cross-user/cross-context leakage.

  • Unbounded cardinality: keying on free-text or full payloads blows up memory; hash long inputs.

  • Collisions from naive concatenation: use clear delimiters or hashing so "a"+"bc" can't equal "ab"+"c".

Good candidates are read-heavy, expensive to produce, and tolerant of some staleness, so reuse pays off far more often than it costs. Poor candidates change constantly, must be perfectly fresh, or are read rarely.

• Good candidates:

  • High read-to-write ratio: read many times between changes, so each cached copy serves many hits.

  • Expensive to compute or fetch: complex joins, aggregations, external API calls.

  • Staleness-tolerant: a few seconds/minutes of lag is acceptable (product catalogs, config).

  • Reusable across requests/users: shared, not unique-per-request.

• Poor candidates:

  • Write-heavy / rapidly changing: cache is invalidated before it earns a hit.

  • Strong freshness/consistency needs: account balances, inventory at checkout.

  • Low reuse / high cardinality: per-request unique data wastes memory with near-zero hit rate.

  • Cheap to recompute: caching adds complexity without saving meaningful cost.

The difference is where the load-on-miss logic lives. In Cache-Aside the application orchestrates the miss: it checks the cache, queries the DB, and populates the cache. In Read-Through the cache itself (via a provider/loader) fetches from the backing store on a miss, transparently to the app.

• Cache-Aside (Lazy Loading):

  • Logic lives in application code.

  • Flow: read cache, on miss read DB, write to cache, return.

  • Pros: simple, cache and DB decoupled (cache outage isn't fatal); cache only holds requested data.

  • Cons: duplicated logic across call sites; first request always pays the miss penalty.

• Read-Through:

  • Logic lives in the caching layer (a configured loader function).

  • Flow: app asks cache; cache returns hit or loads from DB itself, then returns.

  • Pros: app code is simpler and uniform; load logic centralized.

  • Cons: requires cache support/integration; data model must match the loader.

Write-Through writes to the cache and the backing store synchronously in one operation, so the store is always current at the cost of higher write latency. Write-Back writes only to the cache and flushes to the store asynchronously later, giving very low write latency but risking data loss if the cache fails before the flush. For a system where data loss is unacceptable, choose Write-Through.

• Write-Through:

  • Every write hits cache and DB before returning: durable and consistent.

  • Higher write latency; can cache data that's never read (wasted writes).

• Write-Back (Write-Behind):

  • Write to cache, acknowledge immediately, flush to DB in batches/async.

  • Excellent write throughput and latency; absorbs write spikes and coalesces repeated writes.

  • Risk: unflushed data is lost if the cache node crashes before persisting.

• Choice for no-data-loss systems:

  • Write-Through, because durability is guaranteed at commit time.

  • If write-back's performance is needed, it must add a durable log/replication before ack, which erodes its latency advantage.

Use Write-Around when written data is unlikely to be read again soon: writes go straight to the backing store and bypass the cache, so the cache isn't polluted with cold data. The trade-off is that the just-written item is not in cache, so the next read of it is guaranteed to miss.

• When to use it:

  • Write-heavy, read-rarely workloads: logs, audit records, bulk imports.

  • To avoid cache churn: prevents one-time writes from evicting hot, frequently-read keys.

• Impact on miss rate:

  • Raises misses for recently-written items: a read right after a write always misses and loads from DB.

  • But protects overall hit rate by keeping the working set warm, which is the right call when written data is seldom read.

• Common pairing: Write-Around for writes plus Cache-Aside/Read-Through for reads.

All three are write policies differing in whether and when the cache and backing store are updated. Write-Through updates both synchronously; Write-Back updates the cache now and the store asynchronously later; Write-Around updates only the store and skips the cache. Write-Back's danger is durability: data acknowledged but not yet flushed is lost on a cache crash.

• Write-Through:

  • Writes cache + DB together: consistent, durable, but higher write latency.

  • Good when written data is read soon after.

• Write-Back (Write-Behind):

  • Writes cache, acks immediately, flushes to DB async/batched: lowest latency, highest throughput.

  • Data loss risks: crash before flush loses unpersisted writes; batch flush failures can drop a window of updates; reduced durability and a more complex recovery path.

  • Mitigate with replication, persistent journaling, or shorter flush intervals.

• Write-Around:

  • Writes DB only, bypassing cache: avoids polluting cache with cold data.

  • Durable (DB is written), but next read of that item misses.

• Summary: Write-Through favors consistency, Write-Back favors performance at durability risk, Write-Around favors cache efficiency for write-rarely-read data.

LRU evicts the entry that hasn't been accessed for the longest time (recency); LFU evicts the entry accessed the fewest times (frequency). LFU outperforms LRU when a stable set of items is popular over the long term but is occasionally interrupted by bursts of one-off accesses.

• LRU (recency-based):

  • Keeps recently touched items; simple and adapts fast to changing working sets.

  • Weakness: a large scan of unique items evicts genuinely hot data (cache pollution).

• LFU (frequency-based):

  • Keeps items with high historical access counts; resists one-time floods because new items have low counts.

  • Weakness: stale popularity (an item hot last week clings on); needs aging/decay to fix.

• The pattern where LFU wins: A skewed/Zipfian workload with a steady hot set, periodically swept by a batch job or scan reading many distinct keys once. LRU evicts the hot set to hold the scan's junk; LFU protects the hot set.

Cache hit ratio is the fraction of lookups served from cache: hits / (hits + misses). A high hit ratio combined with high latency means the cache is effective at avoiding the backing store but the cache layer itself is slow, pointing to a design or deployment problem rather than a hit-rate problem.

• Definition and measurement:

  • Instrument lookups: increment a hit counter when found, a miss counter when not, and compute the ratio over a rolling window.

  • Measure per tier and per key-class: aggregate ratios can hide a hot keyspace that misses badly.

  • Pair it with latency percentiles (p50/p99), not just averages.

• What high hit ratio + high latency suggests:

  • Network distance: a remote cache (cross-AZ/region) adds round-trip cost on every hit; consider a local in-process tier.

  • Serialization overhead: large values or expensive (de)serialization dominate even when data is found.

  • Contention/hot keys: a few keys serialize on a single shard or lock.

  • Wrong granularity: you cache big blobs and pay to fetch/parse them when only a slice is needed.

• Takeaway: Hit ratio measures effectiveness; latency measures cost-per-hit. Optimize both: a fast miss path can beat a slow hit, so the real metric is overall served latency.

Hit ratio rises with cache size but with diminishing returns: each additional megabyte captures progressively colder data, so you pay linearly in memory for a curve that flattens. Sizing is about finding the knee where extra memory stops meaningfully improving hit ratio.

• The relationship is non-linear:

  • A small cache that fits the hot working set yields most of the benefit; beyond it, you're caching long-tail items accessed rarely.

  • Plotting hit ratio vs. size (a miss-ratio curve) reveals a 'knee' where the slope drops sharply.

• Costs of oversizing:

  • Direct RAM cost and, for in-process caches, GC pressure and longer pauses.

  • More memory can increase eviction-tracking overhead and reduce density for co-located services.

• Costs of undersizing: Thrashing: high eviction churn where items are evicted before reuse, hammering the backing store.

• Practical sizing:

  • Estimate the working-set size, then size at or slightly past the knee of the miss-ratio curve.

  • Account for value-size distribution and metadata overhead, not just item count.

  • Workload matters: high temporal locality reaches the knee quickly; uniform-random access barely benefits at any size.

For sliding-expiration session data, configure a TTL-based eviction with the TTL reset on each access (a 30-minute idle timeout), backed by LRU as a safety valve for memory pressure. The expiry is the primary correctness mechanism; LRU only handles the case where you run out of memory before items naturally expire.

• Primary policy: sliding TTL:

  • Each read/write refreshes the key's expiry to now + 30 minutes, matching 'expires after 30 minutes of inactivity'.

  • Active users stay logged in; idle sessions expire automatically without manual cleanup.

• Secondary policy: LRU eviction under memory pressure:

  • In Redis this maps to volatile-lru (evict least-recently-used among keys that have a TTL), so non-session keys without TTLs aren't touched.

  • LRU aligns naturally with sliding expiry: idle sessions are also the least recently used.

• Why not the alternatives:

  • Pure LFU could keep stale-but-historically-popular sessions and evict newer active ones.

  • Fixed (absolute) TTL would log out active users mid-session.

MRU evicts the item touched most recently, which wins when the most recently used item is the least likely to be needed again soon: namely sequential or cyclic scans over a dataset larger than the cache. LRU is exactly wrong there because it keeps recent items that won't recur and evicts the older ones you're about to loop back to.

• The scan / looping pattern:

  • Reading a table sequentially repeatedly: once you read a record you won't touch it again until the next full pass.

  • If the dataset is bigger than the cache, LRU evicts the oldest entries, which are precisely the ones the next cycle reaches first, giving near-zero hits.

• Why MRU helps here: By discarding the just-used item, it preserves older cached blocks that the cyclic scan will revisit sooner, salvaging some hits.

• Caveat: MRU is a niche policy: for the common case of temporal locality (recently used = likely reused), MRU is the worst choice. It's used adaptively, e.g., systems that detect scan patterns and switch policies.

Both control reuse, but they differ on whether storage is allowed. no-cache lets the browser store a copy but requires revalidation with the origin before using it; no-store forbids storing the response at all. So no-cache is the one that keeps a copy.

• no-cache:

  • Cache may store the response but must revalidate (conditional request with ETag/If-Modified-Since) before serving it.

  • If the origin returns 304 Not Modified, the stored copy is reused, saving bandwidth. Good for content that changes unpredictably but is often unchanged.

• no-store:

  • Nothing is written to any cache (browser or intermediary); every request goes fully to the origin.

  • Used for sensitive data (banking pages, personal info) where leaving a copy on disk is unacceptable.

• Common confusion: The name no-cache sounds like 'don't store' but actually means 'don't use without checking'.

The Vary header tells caches that the response depends on certain request headers, so a cached entry is only reused when those headers match. By varying on the header that distinguishes device type, a cache will treat the mobile and desktop responses as separate entries instead of serving one to the other.

• How it works:

  • A cache key normally is just the URL; Vary extends the key to include the listed request headers' values.

  • Example: Vary: User-Agent means requests with different User-Agent values get distinct cached copies.

• Mobile vs. desktop: If the server renders different HTML by device and sets Vary appropriately, a mobile request won't match the desktop cache entry, so it fetches/serves the mobile version.

• Practical caution:

  • Varying on User-Agent is high-cardinality and explodes cache entries; many sites instead vary on a normalized hint like Vary: Sec-CH-UA-Mobile or serve responsive markup so no variation is needed.

  • Common companions: Vary: Accept-Encoding (gzip vs. identity) and Vary: Accept-Language.

An ETag is an opaque validator (often a hash or version) the server attaches to a response identifying that exact representation. On the next request the client sends it back in If-None-Match; if the resource is unchanged the server replies 304 Not Modified with no body, so the client reuses its stored copy and only header bytes cross the wire.

• The conditional GET flow:

  • Initial response includes ETag: "abc123" and the full body.

  • Client revalidates by sending If-None-Match: "abc123".

  • Server compares: match means 304 Not Modified (empty body); mismatch means 200 OK with the new body and a new ETag.

• Why it saves bandwidth:

  • A 304 carries only headers, avoiding retransmission of an unchanged (possibly large) payload.

  • The round trip still happens, so it saves bytes and rendering work, not the request itself (unlike a fresh max-age hit, which avoids the trip entirely).

• Strong vs. weak:

  • Strong ETags mean byte-for-byte identical; weak ETags (W/"...") mean semantically equivalent, useful when minor encoding differs.

  • More precise than Last-Modified, which has 1-second granularity and can't detect content reverts.

The public directive lets any cache (including shared/intermediary caches like CDNs and proxies) store the response, while private restricts storage to the end user's browser cache only. Getting this wrong on authenticated responses can leak one user's data to another.

• public:

  • Any cache may store it, even shared ones serving many users.

  • Appropriate for non-personalized assets: CSS, JS, images, public API responses.

• private:

  • Only the user-agent's private cache may store it; shared caches must not.

  • Use for personalized/authenticated content (account pages, user-specific data).

• Why the security distinction matters:

  • If an authenticated response is marked public, a shared proxy/CDN can cache user A's response and serve it to user B.

  • For sensitive data, prefer Cache-Control: private, no-store or no-cache to prevent leakage entirely.

• Note: private is about who may store, not encryption; it is not a confidentiality guarantee on its own.

Both are validators, but Last-Modified is a timestamp (1-second resolution) while ETag is an opaque content fingerprint. ETags are more precise and are the preferred validator when both are available.

• Last-Modified:

  • Sends a date; client revalidates with If-Modified-Since.

  • Limitations: 1-second granularity misses sub-second changes, and content that's rewritten without changing meaning still looks "modified."

• ETag:

  • Sends an opaque token (often a hash); client revalidates with If-None-Match.

  • Detects any content change precisely and supports strong vs. weak semantics.

• Precedence: If both are present, caches use the ETag for validation; Last-Modified can serve as a fallback heuristic.

Freshness tells a cache how long it may reuse a response with no server contact at all; validation tells it how to check with the server whether a stale copy is still good. A browser sends a Conditional GET only after the response is stale and it holds a validator.

• Freshness (Cache-Control: max-age / Expires): While fresh, the cache serves directly with zero network round-trip.

• Validation (ETag / Last-Modified): Stored validators let the cache ask "has this changed?" without re-downloading the body.

• When the Conditional GET fires:

  • The entry is stale (max-age elapsed or no-cache forces revalidation) AND a validator exists.

  • It sends If-None-Match and/or If-Modified-Since; server replies 304 (reuse) or 200 (new body).

Stale-While-Revalidate (SWR) lets a cache serve a slightly stale response instantly while it asynchronously fetches a fresh copy in the background: the user never waits for revalidation, and the next request gets the updated data.

• How it works:

  • Configured via Cache-Control: max-age=600, stale-while-revalidate=30.

  • During the SWR window after expiry, the stale entry is returned immediately and a background revalidation refreshes it.

• Latency benefit: The slow round-trip happens off the critical path, so perceived latency stays near a cache hit.

• Eventual freshness: Each request triggers refresh, so data converges to current within one cycle; beyond the SWR window it falls back to blocking revalidation.

• Trade-off: you accept bounded staleness in exchange for speed, so don't use it where strict correctness matters.

Cache warming is proactively populating a cache with likely-needed data before real traffic arrives, so the first users don't all hit a cold cache and hammer the origin. It's necessary whenever a cold cache would cause a latency spike or an overload (a stampede) on the backing store.

• The problem it solves:

  • A cold cache means every early request misses and falls through to the database, spiking latency and load.

  • Prevents a thundering herd where many concurrent misses recompute the same value.

• When to warm:

  • After a deploy/restart that flushed an in-memory cache.

  • Before a predictable traffic surge (product launch, sale, scheduled event).

  • After a cache cluster failover or scaling event that starts nodes empty.

• How: Replay known hot keys, run a script to pre-fetch top queries, or shadow real traffic before cutover.

TTL eviction is time-based: a key is removed when its expiry passes regardless of memory. Max-memory eviction is pressure-based: when the store hits its memory limit, it evicts keys (per a policy like LRU/LFU) to make room. And yes, a key can absolutely be evicted before its TTL expires.

• TTL eviction:

  • Each key has an expiry; removed lazily (on access) or by a background sweeper once expired.

  • Bounds staleness, not memory usage.

• Max-memory eviction:

  • Triggered when maxmemory is reached; a policy decides victims (e.g. allkeys-lru, allkeys-lfu, volatile-ttl).

  • Bounds memory, independent of whether keys have expired.

• Can a key die before its TTL?:

  • Yes: under memory pressure an LRU/LFU policy can evict a still-valid key. Policies like noeviction instead reject writes once full.

  • Implication: callers must never assume a cached key is present just because its TTL hasn't passed.

There is no universal TTL: you balance data freshness requirements against backend load, then validate empirically. The optimal TTL is the longest duration you can serve stale data without harming correctness or user trust.

• Start from tolerance for staleness: Ask how wrong the data can be: stock prices need seconds, a user's profile name can be minutes, a country list can be hours or days.

• Weigh against backend cost: Expensive-to-compute or rarely-changing data justifies a longer TTL; cheap, fast queries gain little from aggressive caching.

• Consider update frequency vs read frequency: High read / low write data is the sweet spot for long TTLs; if writes are frequent, prefer event-based invalidation over a long TTL.

• Validate with metrics: Tune by watching hit ratio and origin load: raise TTL if hit ratio is low and staleness is acceptable; lower it if users see stale data.

• Add jitter: Randomize TTLs slightly so many keys don't expire simultaneously and cause a stampede.

Lazy expiration removes an expired entry only when it's next accessed, while active expiration uses a background process to sweep and delete expired entries. Real systems (like Redis) combine both to balance CPU cost against memory reclamation.

• Lazy (passive) expiration:

  • On read, check the timestamp; if expired, delete and treat as a miss.

  • Cheap (no background work) but expired keys that are never read again sit in memory forever, wasting space.

• Active (eager) expiration:

  • A background job periodically samples or scans keys and evicts expired ones.

  • Reclaims memory promptly but costs CPU, and a full scan can be expensive at scale (so Redis samples a random subset).

• The trade-off in one line: Lazy saves CPU but leaks memory; active reclaims memory but spends CPU. Combining them gives bounded memory with low overhead.

TTL expiration is time-driven and approximate: the cache eventually drops stale data on its own. Explicit invalidation is event-driven and precise: a write to the source actively removes or updates the cached copy immediately. They are complementary, and using both gives precision plus a safety net.

• Time-based (TTL) expiration: Simple and self-healing: no coordination needed, but allows a staleness window equal to the TTL.

• Explicit (event-based) invalidation:

  • On a data change, the writer deletes or updates the key, so reads reflect changes instantly.

  • More precise but harder: you must reliably catch every mutation path, and missed events leave stale data forever.

• Why combine them: Use explicit invalidation for fast correctness on known writes, and a moderate TTL as a backstop so any missed invalidation eventually self-corrects.

Cache penetration is when requests repeatedly ask for keys that don't exist in the cache or the database, so every request misses the cache and hammers the backend. It differs from a normal miss because a normal miss eventually populates the cache, whereas a non-existent key never gets cached and so every request keeps falling through, which makes it a denial-of-service vector.

• Standard miss vs penetration: A normal miss loads the value from the DB and caches it, so subsequent reads hit. With penetration the DB returns nothing, nothing is cached, and the next identical request misses again.

• Negative caching: Cache the "not found" result (often with a short TTL) so repeated lookups for the absent key are served from cache, not the DB.

• Bloom filter:

  • Keep a probabilistic set of keys that actually exist; if the filter says a key is absent, reject it before touching cache or DB.

  • It can have false positives (lets a few through) but never false negatives (never blocks a real key).

• When to use which: Negative caching is simple for a few repeated bad keys; a bloom filter scales better against large or random sets of invalid keys (e.g. malicious ID scanning).

A cache avalanche happens when a large set of keys expire at the same moment (or the cache layer goes down), so a flood of requests all miss simultaneously and slam the database at once, often crashing it. Jitter spreads those expirations out over time so misses arrive gradually instead of in one spike.

• The avalanche scenario:

  • Many keys are loaded together (e.g. a warm-up job) with the same TTL, so they all expire in the same second.

  • Every read for those keys then misses at once and falls through to the DB, which sees a sudden multiplied load and tips over.

• Why jitter helps:

  • Instead of a fixed TTL, set TTL = base + random(0, spread) so expirations scatter across a window.

  • Misses become a smooth trickle the DB can absorb rather than a synchronized cliff.

• Complementary defenses:

  • Cache replication / HA so a node failure doesn't expire everything at once.

  • Request coalescing or locks so only one rebuild per key hits the DB.

Both flood the database, but for opposite reasons: penetration is requests for data that doesn't exist (so it's never cacheable and always misses), while avalanche is many valid cached keys expiring at once (the cache momentarily can't serve them). Penetration is a key-validity problem; avalanche is a timing problem.

• Cache Penetration:

  • Queries for keys that exist in neither cache nor DB (often malicious, e.g. random/nonexistent IDs).

  • Nothing ever gets cached, so every request reaches the DB.

• Cache Avalanche:

  • Valid keys all expire (or the cache fails) simultaneously, so a burst of misses hits the DB.

  • The data is cacheable; the issue is everything refilling at the same time.

• How mitigations differ:

  • Penetration: validate inputs, use a Bloom filter to reject keys that can't exist, and negative-cache the misses.

  • Avalanche: add TTL jitter, replicate the cache for HA, and use locks/coalescing to rebuild hot keys safely.

Negative caching means caching the fact that a lookup returned nothing (a "not found" or null result), usually with a short TTL. It protects the DB against cache penetration: repeated requests for a nonexistent key get answered from the cache instead of hammering the database every time.

• The problem it solves:

  • Normally a miss for a nonexistent key falls through to the DB; if the answer is "doesn't exist," nothing gets cached and the next identical request misses again.

  • Attackers (or buggy clients) can exploit this by repeatedly requesting missing IDs.

• How it works: Store a sentinel value (e.g. NULL or an empty marker) so subsequent lookups hit the cache and return "not found" instantly.

• Use a short TTL: Keep negative entries short-lived so that when the data is eventually created, clients see it without a long staleness window.

• Watch the memory cost: Caching unbounded random missing keys can itself bloat the cache; pair with a Bloom filter for high-volume penetration attacks.

Consistent hashing maps both cache nodes and keys onto the same circular hash space (a ring), and each key is owned by the next node clockwise. When a node is added or removed, only the keys in that node's immediate arc need to move, so the vast majority of keys keep mapping to the same node and stay cached.

• The ring model:

  • Hash node identifiers and keys into the same space (e.g. 0 to 2^32-1) arranged as a circle.

  • A key belongs to the first node encountered walking clockwise from the key's position.

• Why churn is minimal:

  • Removing a node reassigns only its keys to the next node clockwise; all other keys are untouched.

  • Adding a node only steals the slice of keys between it and its predecessor.

  • On average only about K/N keys remap (K keys, N nodes), versus nearly all keys with mod-N hashing.

• Virtual nodes: Each physical node is placed at many points on the ring to even out load and avoid one node inheriting a huge arc.

Consistent hashing is a technique that places keys and nodes on a hash ring so each key is owned by the next node clockwise, ensuring that changing the node count only remaps a small fraction of keys. It's preferred over "mod N" because mod-N ties the mapping to the exact node count, so adding or removing one node reshuffles almost every key and effectively wipes the cache.

• The problem with mod N:

  • Placement is hash(key) % N, so when N changes the modulus changes for essentially all keys.

  • Nearly every key now maps to a different node: a mass cache miss and a DB stampede on each scaling event.

• How consistent hashing fixes it:

  • Both keys and nodes hash onto a ring; only keys between the changed node and its neighbor move.

  • Roughly K/N keys remap instead of all K, so most of the cache stays warm.

• Practical refinements:

  • Virtual nodes spread each physical node across many ring positions for balanced load.

  • Used by Dynamo, Cassandra, and many distributed caches for exactly this elasticity.

Sharding splits data across nodes so each node holds a subset; replication copies the same data to multiple nodes. Sharding adds storage capacity (and write scale); replication adds read throughput (and availability).

• Sharding (partitioning):

  • Each key maps to one shard (via hashing or a slot map), so the dataset is divided rather than duplicated.

  • Scales total storage and aggregate write/read load because work spreads across nodes.

  • Trade-off: cross-shard operations are hard, and a lost shard loses its slice of data unless also replicated.

• Replication:

  • A primary's data is mirrored to one or more replicas; reads can fan out across replicas.

  • Improves read throughput and provides failover/high availability.

  • Does NOT add storage capacity (every replica holds the full copy) and async replicas can serve slightly stale reads.

• They are complementary: production clusters typically shard for capacity and replicate each shard for reads and resilience (e.g. Redis Cluster).

Choose Redis when you need rich data structures, persistence, or built-in clustering and replication; choose Memcached when you want a dead-simple, multi-threaded, pure key/value cache for raw throughput with minimal features.

• Pick Redis for:

  • Rich types: lists, sets, sorted sets, hashes, bitmaps, streams (good for leaderboards, queues, rate limiting).

  • Persistence (RDB/AOF), replication, and native clustering for HA and durability.

  • Extras like pub/sub, Lua scripting, and atomic server-side operations.

• Pick Memcached for:

  • Simple, large-scale string/object caching where you only need GET/SET.

  • Multi-threaded design uses many cores per node well for high concurrent throughput.

  • Slab allocation can be efficient for uniform object sizes and predictable memory.

• Architectural takeaway: Redis is closer to a Swiss-army datastore that can also cache; Memcached is a focused cache. Default to Redis unless you specifically want minimalism and per-node thread scaling.

Cache invalidation is the problem of removing or updating cached data at exactly the right time so callers never see stale values while still getting the speed benefit of caching. It is famously hard because there is no perfect signal for 'when' to invalidate without coupling the cache to every possible source of change.

• Why it's hard:

  • Timing: invalidate too early and you lose hit rate; too late and you serve stale data.

  • Distributed state: many nodes (and multi-level caches) each hold copies that must all be updated atomically, which is impossible without coordination.

  • Dependency tracking: one underlying change can invalidate many derived/aggregated keys that are hard to enumerate.

  • Races: a read can repopulate a key with stale data right after an invalidation (concurrent miss + write).

• Common mitigations:

  • TTL/expiry as a coarse self-correcting bound.

  • Write-through / write-behind to update on the write path.

  • Event-driven invalidation (CDC, pub/sub) and versioned keys to sidestep deletes.

• The 'two hard things' quip (Phil Karlton): 'cache invalidation and naming things' captures that correctness here resists simple, general solutions.

A versioned key embeds a version into the cache key (e.g. user:v2:123) so changing the version instantly makes all old entries unreachable. New reads miss and repopulate under the new key, giving you an atomic, bulk invalidation without deleting anything.

• How it helps during schema/deploy changes:

  • Bumping the version (config or app constant) effectively invalidates the entire namespace at once: no scanning or mass DELETE needed.

  • Avoids serving a v2 deployment data shaped for v1, preventing deserialization errors and stale shapes.

• Operational benefits:

  • No race window: old and new keys coexist, so in-flight requests on old code still hit v1 while v2 builds up v2.

  • Easy rollback: revert the version and the old cache is live again.

  • Old keys simply age out via TTL/eviction, so no cleanup step is required.

• Cost: A temporary cold-cache spike after the bump, and extra memory while both versions linger.

Strong consistency means every read returns the most recent write no matter which node serves it; eventual consistency means nodes may briefly return stale data but converge to the same value once updates propagate. The trade is correctness guarantees versus latency and availability.

• Strong consistency:

  • Reads reflect the latest committed write everywhere, often via synchronous replication or quorum (write and read majorities overlap).

  • Higher write latency and reduced availability during partitions (CAP: favors C over A).

• Eventual consistency:

  • Writes propagate asynchronously, so different nodes can temporarily disagree (a staleness window).

  • Lower latency and higher availability: ideal for caches where slight staleness is acceptable (favors A over C).

• Choosing for a cache: Caches usually accept eventual consistency (bounded by TTL) for speed; use strong consistency only for data that must never be stale (e.g. balances, inventory counts), accepting the performance cost.

In cache-aside the cache and the database are updated separately, so there is always a brief window where they disagree; eventual consistency means the cache will converge to the DB value (after invalidation or TTL), but reads in between may be stale.

• How cache-aside creates the gap:

  • Read: app checks cache; on miss, loads from DB and populates the cache.

  • Write: app updates the DB and then invalidates (or updates) the cache as a second, non-atomic step.

• Where staleness comes from:

  • Between the DB write and the cache invalidation, readers see the old cached value.

  • A race: a concurrent miss can repopulate the cache with the old DB value just before the write commits.

• How it converges (the 'eventual' part):

  • Once invalidation lands or the TTL expires, the next read reloads the fresh value.

  • TTL is the upper bound on staleness even if an invalidation message is lost.

• Practical guardrails: keep TTLs modest, delete (not update) the cache on writes, and use techniques like delayed double-delete to close races.

Virtual nodes are multiple positions on the hash ring that each physical server owns, instead of just one: they smooth out load distribution and reduce disruption when nodes join or leave.

• The problem they solve:

  • With one point per server, hashing rarely spaces servers evenly, so some get large arcs of the ring (and thus more keys) than others: load is lopsided.

  • When a server dies, its entire arc shifts to a single neighbor, overloading it.

• How vnodes fix it:

  • Each physical node maps to many points (e.g. 100-200), so the law of large numbers evens out the arcs each server owns.

  • When a node fails, its many small arcs redistribute across many remaining nodes, not just one.

• Bonus: heterogeneity: Give a more powerful machine more vnodes to make it take proportionally more load.

Serialization sits on the hot path of every cache read and write, so its CPU cost and byte size directly shape latency, throughput, and memory: the wrong format can erase the benefit of caching.

• It runs on every operation: Encode on write, decode on read: slow (de)serialization adds latency to the very requests caching was meant to speed up.

• Payload size drives multiple costs: Smaller bytes mean less network transfer, more entries per GB of cache (higher hit rate), and faster I/O.

• Format trade-offs:

  • JSON: human-readable and debuggable, but verbose and slow to parse.

  • Protobuf/binary: compact and fast, with schema evolution, but opaque and needs codegen/tooling.

• Schema and compatibility: A cache is long-lived shared state, so versioning matters: a format with explicit schema evolution avoids deserialization failures when models change.

• Rule of thumb: High-throughput internal paths favor binary (Protobuf); debuggability and low-traffic paths can tolerate JSON.

The dual-write problem is that updating two systems (cache and database) in separate, non-atomic steps can fail or interleave so they end up inconsistent: one write succeeds and the other doesn't, or concurrent writers apply updates in different orders. There is no perfect fix; you reduce the window and make the DB the source of truth.

• Why it happens:

  • No shared transaction: a crash between the DB write and cache write leaves them divergent.

  • Race condition: two updates plus a stale repopulation can reorder, leaving an old value cached.

• Mitigations:

  • Invalidate, don't update: write DB then delete the key (let the next read repopulate). Deletes are idempotent and avoid writing a stale value.

  • Always treat the DB as the source of truth; the cache is disposable.

  • Set TTLs so any inconsistency self-heals within a bounded window.

  • For tighter consistency, drive the cache from the DB's change log (CDC / outbox), giving a single ordered stream of updates instead of two independent writes.

  • Guard against stale repopulation races with versioning or a read-recompute-then-set with checks.

Pushing caching into a library or sidecar (a Read-Through cache) centralizes the read/populate logic so application code just asks for data and never sees the cache miss machinery: this reduces duplication, bugs, and inconsistency.

• Separation of concerns: The app calls one read API; the cache layer handles miss detection, loading from the source, and storing. Business logic stays clean.

• Consistency across callers: Manual cache-aside repeated in many places drifts: each call site may use different keys, TTLs, or forget to populate. A library enforces one policy.

• Built-in protections: A shared layer can add request coalescing/single-flight, stampede protection, and metrics once, for everyone.

• Sidecar advantages: Language-agnostic: many services share the same caching behavior without each reimplementing it.

• Trade-off: You give up fine-grained control over exactly when/what to cache, and a sidecar adds a network hop and operational surface.