Distributed Computing Expert

1 / 96

What are the Fallacies of Distributed Computing? Name at least three and explain how ignoring them leads to system failure.

Select the correct answer

1

Assuming servers are stateless, caches are warm, and queues are unbounded.

2

Assuming clocks are synchronized, retries are free, and nodes never crash.

3

Assuming the network is reliable, latency is zero, and bandwidth is infinite.

4

Assuming data is consistent, writes are atomic, and every read is cheap.

Explain the difference between 'Horizontal Scaling' and 'Vertical Scaling' from a state-management perspective.

Select the correct answer

1

Vertical scaling keeps state on one machine; horizontal scaling requires distributing or externalizing state.

2

Vertical scaling externalizes state to a store; horizontal scaling keeps state local to each node.

3

Vertical scaling partitions state by key; horizontal scaling stores all of the state in memory.

4

Vertical scaling replicates state across nodes; horizontal scaling keeps a single shared local copy.

What is an RPC, and what challenges arise from treating a remote call like a local function call?

Select the correct answer

1

It invokes a procedure via shared memory; unlike sockets it avoids latency, retries, and any serialization.

2

It invokes a procedure on a remote machine; unlike local calls it faces latency, partial failure, and serialization.

3

It invokes a procedure inside the same process; unlike threads it avoids locks, deadlocks, and shared memory.

4

It invokes a procedure on a remote machine; like local calls it is instant, reliable, and always succeeds.

What is the difference between the request/response and publish/subscribe coordination paradigms?

Select the correct answer

1

Request/response broadcasts to all listeners; pub/sub sends one targeted message and waits for a single acknowledgment.

2

Request/response uses a message broker for buffering; pub/sub connects sender and receiver directly for each reply.

3

Request/response is direct and one-to-one; pub/sub is decoupled and one-to-many via a broker without known receivers.

4

Request/response is decoupled and one-to-many; pub/sub is direct one-to-one requiring the sender to await a reply.

Why do we build distributed systems in the first place: what problems do scalability, fault tolerance, low latency, and geographic distribution solve?

Select the correct answer

1

To simplify deployment, reduce total code, and remove the need for monitoring or coordination across nodes.

2

To exceed a single machine's limits, survive component failures, and serve users nearer to their location.

3

To lower hardware cost, avoid concurrency issues, and ensure every request is processed on one central server.

4

To eliminate network overhead, guarantee strong consistency, and remove the need for any data replication.

What is partial failure, and why does it make distributed systems fundamentally harder to reason about than single-machine programs?

Select the correct answer

1

Some components fail while others keep running, and a caller often cannot tell whether a remote call succeeded, failed, or was merely delayed.

2

A running program crashes completely, forcing every node in the cluster to restart together so that they can regain a consistent state.

3

Data gets written only partly to disk, so each node must keep a write-ahead log to recover the missing portion after a reboot occurs.

4

Individual CPU cores fail on their own, so a single machine has to include redundant processors to keep executing the current workload.

Explain the Two Generals Problem. Why can two nodes never be 100% certain they have reached a common agreement over an unreliable network?

Select the correct answer

1

One node may behave maliciously, so trust between the two parties can never be established.

2

Every message including acknowledgments can be lost, so no finite exchange guarantees mutual certainty.

3

Network latency varies widely, so the two nodes cannot synchronize clocks to act at once.

4

Messages can arrive out of order, so the two nodes cannot agree which attack time is correct.

What is the difference between synchronous and asynchronous communication models in distributed systems, and what are the trade-offs?

Select the correct answer

1

Asynchronous callers block for a reply, giving low latency; synchronous sends to a broker for higher resilience.

2

Synchronous callers block for a reply, giving simplicity but tight coupling; async decouples at the cost of complexity.

3

Synchronous callers use many threads, giving parallelism but high memory; async uses one thread with lower throughput.

4

Synchronous callers never wait for replies, giving speed; async always waits, giving guaranteed ordering and delivery.

Why do stateless services scale more easily than stateful ones, and how do we handle state when scaling horizontally?

Select the correct answer

1

They replicate session data to every node, so requests route freely; state stays cached inside each instance.

2

They store all state on the client only, so servers coordinate; state is synchronized between nodes on write.

3

They keep no local session data, so any instance serves any request; state is externalized to shared stores.

4

They pin each client to one node, so requests are sticky; state is held in local server memory only.

What is the difference between Strong Consistency and Eventual Consistency? When would you choose one over the other?

Select the correct answer

1

Strong: reads always return the latest write; eventual: replicas converge later and reads can be stale; pick strong for critical data, eventual for scale.

2

Strong: writes are ordered per client only; eventual: writes are globally ordered; pick strong for availability needs, eventual for strict correctness needs.

3

Strong and eventual both guarantee latest reads but differ in latency; pick strong when latency matters, eventual when throughput matters more overall.

4

Strong: replicas converge over time and reads may be stale; eventual: reads always return the latest write; pick strong for scale, eventual for critical data.

Explain 'Read-Your-Writes' consistency. How would you implement it in a system with asynchronous replication?

Select the correct answer

1

After a client writes a value, its own later reads always reflect that write; route the client's reads to replicas that have its last write.

2

Writes issued by one client are applied in the same order at every replica; push all writes through a single ordered global queue to enforce this.

3

After any write anywhere, every client immediately sees the newest value; make all replicas synchronously acknowledge each write before returning.

4

A client always observes the most recent write made by other clients; lock each record during reads so no concurrent update is missed here.

Explain the trade-off between Latency and Consistency in a system that is not currently experiencing a network partition.

Select the correct answer

1

Even without a partition, stronger consistency needs replica coordination that adds latency, so you trade lower latency against weaker consistency.

2

Even without a partition, weaker consistency needs more coordination and adds latency, so you trade higher latency for stronger consistency guarantees.

3

Without a partition there is no trade-off, since replicas are reachable and can always return strongly consistent reads with no extra latency cost.

4

Without a partition the only trade-off is durability versus latency, because consistency is guaranteed automatically whenever all nodes are online.

What is the difference between Linearizability and Eventual Consistency? When is Eventual Consistency 'not enough'?

Select the correct answer

1

Linearizability makes reads see the newest write in real time; eventual only guarantees convergence; eventual fails when stale reads cause wrong behavior.

2

Linearizability and eventual both prevent stale reads but differ in cost; eventual fails only when write throughput exceeds the cluster's replication limit.

3

Linearizability only guarantees eventual convergence; eventual guarantees real-time recency; eventual fails when replicas are allowed to diverge for too long.

4

Linearizability orders transactions while eventual orders single writes; eventual fails when a system needs multi-object serializable isolation guarantees.

What are monotonic reads and monotonic writes consistency, and what anomalies do they prevent?

Select the correct answer

1

Monotonic reads prevent seeing older data after newer; monotonic writes keep a process's writes ordered.

2

Monotonic reads ensure the latest write is always read; monotonic writes prevent two clients writing the same key.

3

Monotonic reads guarantee all replicas return identical values; monotonic writes force every node to write synchronously.

4

Monotonic reads block reads during writes; monotonic writes serialize all writes across the entire cluster globally.

Compare Strong Consistency vs. Causal Consistency. When would you prefer the latter?

Select the correct answer

1

Strong consistency lets replicas diverge and reconcile later automatically; prefer it when clients can tolerate reading somewhat stale values for a while.

2

Strong consistency orders only causally related operations locally per node; prefer it when concurrent writes never touch the same overlapping keys at all.

3

Causal consistency orders only causally related operations without global coordination; prefer it when you want ordering yet high availability and low latency.

4

Causal consistency requires every replica to agree synchronously on a total order; prefer it when correctness matters far more than availability or latency.

What is the difference between Linearizability and Serializability, and can a system be both?

Select the correct answer

1

Linearizability and serializability are the very same guarantee under different names, so asking whether a system can be both is essentially meaningless.

2

Linearizability concerns multi-object transaction ordering; serializability is a recency guarantee on single objects; no system can provide both together.

3

Linearizability is a real-time recency guarantee on single objects; serializability concerns transactions over many objects; a system can be both.

4

Linearizability guarantees eventual convergence of replicas; serializability guarantees isolation of transactions; combining both is possible but never useful.

What is the difference between Linearizability and Sequential Consistency? Which one is harder to achieve and why?

Select the correct answer

1

Both need one total order, but sequential consistency also respects real-time order; it is harder because clocks must be perfectly synchronized across nodes.

2

Both need one total order, but linearizability also respects real-time order; it is harder because it constrains even non-overlapping operations globally.

3

Linearizability allows reordering per process while sequential does not; sequential is harder because it demands a single global wall-clock timestamp source.

4

Neither requires a total order of operations at all; linearizability is harder only because it additionally requires transactions to be fully serializable.

Explain the CAP theorem. In a real-world network partition, why is it technically impossible to choose both Consistency and Availability? Can you give an example of a 'CP' vs. an 'AP' system?

Select the correct answer

1

During a partition writes are queued and replayed later, giving both C and A; DynamoDB is CP while etcd is an AP system example.

2

During a partition you can keep both if replication is fast enough; MongoDB is CP while Redis stays fully AP by default here.

3

During a partition nodes cannot sync, so you either reject requests (CP) or serve stale data (AP); ZooKeeper is CP, Cassandra AP.

4

During a partition consistency stays automatic, so the only real choice is between availability and partition tolerance; HBase is AP.

What is a 'Quorum' in a distributed system, and how do the values of N, R, and W affect consistency and availability?

Select the correct answer

1

N is the number of reads, R and W the retry counts; smaller values always improve both consistency and availability at once.

2

N is the write count, R and W the read and node totals; balancing them has no effect on consistency, only throughput limits.

3

N is the total nodes, R and W the replication delays; only W affects consistency while R alone controls cluster availability.

4

N is the number of replicas, R and W the nodes needed per read or write; larger R and W favor consistency over availability.

Explain the relationship R + W > N. If I want to increase read throughput in a distributed database, how should I tune these variables, and what is the trade-off regarding consistency?

Select the correct answer

1

Lower R and raise W for faster reads; but if R + W ≤ N you can lose the strong-consistency guarantee.

2

Lower both R and W for faster reads; consistency is unaffected because the overlap between replicas is always preserved anyway.

3

Raise R and lower W for faster reads; this keeps strong consistency while also making every single write far cheaper to perform.

4

Raise both R and W for faster reads; the only trade-off is extra storage cost, with no real impact on read consistency at all.

Why is 'Partition Tolerance' generally considered non-negotiable in distributed systems?

Select the correct answer

1

Networks inevitably drop or delay messages, so partitions will happen and cannot be prevented, systems must tolerate them.

2

Modern data-center networks are reliable enough that partitions are rare, but tolerance is kept purely for regulatory compliance needs.

3

Partition tolerance is what makes replication possible, so without it a system could never store more than one copy of any data.

4

Partition tolerance guarantees consistency and availability together, so dropping it would break both of the other properties at once.

Explain the PACELC theorem. How does it extend the CAP theorem, and what does it tell us about a system when there is no network partition?

Select the correct answer

1

It replaces CAP entirely by proving that consistency and availability can always coexist whenever the network is fully connected.

2

It states partitions never occur in practice, so systems only ever balance raw performance against the durability of stored data.

3

It extends CAP: if Partitioned choose A or C, Else choose Latency or Consistency; normally it is a latency-consistency tradeoff.

4

It extends CAP: during partitions choose A or C, Else choose Latency or Availability; normally it is a latency-availability tradeoff.

What is the Byzantine Generals Problem, and how does it differ from a simple 'fail-stop' or 'crash' failure? When would a software engineer actually need to care about Byzantine Fault Tolerance (BFT)?

Select the correct answer

1

Byzantine nodes may act arbitrarily or maliciously, unlike crash faults where nodes simply stop; BFT matters in trustless systems like blockchains.

2

Byzantine faults are just slow nodes, unlike crash faults that go silent; BFT matters mainly when tuning timeouts in a local cluster network.

3

Byzantine faults occur only on disk, unlike crash faults on the network; BFT matters when checksumming files inside a single-machine database.

4

Byzantine faults are recoverable restarts, unlike permanent crash faults; BFT matters whenever replicas need to elect a leader among themselves.

What is graceful degradation, and how does a distributed system stay partially functional when components fail?

Select the correct answer

1

The system keeps core features running at reduced capability instead of failing entirely

2

The system shuts down cleanly to preserve data integrity whenever a critical component fails

3

The system replicates all data synchronously so that no functionality is ever lost in a failure

4

The system automatically restarts failed components to restore full service before users notice

What is a Phi-Accrual Failure Detector, and how does it differ from a simple heartbeat timeout?

Select the correct answer

1

It outputs a continuous suspicion value from heartbeat-arrival statistics, instead of a fixed binary alive-or-dead timeout.

2

It replaces heartbeats with acknowledgements from every peer, producing a continuous value only once the whole cluster fully agrees.

3

It sends heartbeats more frequently than usual to detect failures faster, but still returns a simple alive-or-dead binary result.

4

It waits for a fixed timeout like a heartbeat but averages several rounds, giving the same binary output after multiple missed beats.

What is a 'Lease' in distributed systems, and how does it improve upon a standard heartbeat for failure detection?

Select the correct answer

1

A periodic signal that nodes send to peers to confirm that they are still alive and reachable

2

A lock held indefinitely by one node until it voluntarily chooses to release the shared resource

3

A time-bound grant of a right that expires unless renewed, so failure needs no consensus

4

A quorum vote that elects a fresh leader whenever the current leader stops answering requests

What is a cascading failure in a distributed system, and what techniques like load shedding help contain it?

Select the correct answer

1

A failure that overloads surviving nodes and spreads; load shedding drops excess requests to protect them

2

A failure where one node simply crashes; automatic failover reroutes its traffic to healthy replicas

3

A failure caused by a network partition; quorum reads are used to keep the whole system consistent

4

A failure that propagates via corrupted replication; checksums are used to detect and halt its spread

What are the omission and timing failure models, and how do they sit between crash and Byzantine failures?

Select the correct answer

1

Omission and timing both mean a node sends arbitrary malicious messages, making them worse than Byzantine

2

Omission and timing describe clean halting failures that are strictly simpler to handle than crash faults

3

Omission drops messages and timing responds outside bounds; both are worse than crash, milder than Byzantine

4

Omission means a node halts silently and timing means it sends corrupted values; both equal Byzantine faults

What is redundancy, and how does replication contribute to availability versus consistency?

Select the correct answer

1

Redundancy means compressing repeated data; replication improves both availability and consistency automatically with no coordination overhead at all.

2

Redundancy means splitting data across nodes; replication raises consistency but reduces availability because every read must contact all replicas.

3

Redundancy means deleting duplicate copies; replication lowers availability but guarantees strong consistency because only one copy is ever writable.

4

Redundancy means keeping multiple copies; replication raises availability and durability but creates consistency challenges since copies may diverge.

Explain how Consistent Hashing works. How does it minimize data movement when a node is added or removed compared to a simple hash(key) mod N approach?

Select the correct answer

1

Consistent hashing rehashes every key with a new modulus, spreading them uniformly across all nodes

2

With mod N, adding a node remaps only one key while consistent hashing remaps every key evenly

3

Consistent hashing stores several copies of each key, so removing a node ends up moving no data at all

4

Keys and nodes map onto a ring, so only keys near the changed node move rather than nearly all keys

What are Virtual Nodes in consistent hashing, and what problem do they solve regarding data distribution?

Select the correct answer

1

Each virtual node caches frequently accessed keys in memory to reduce latency for hot partitions

2

Each virtual node replicates a key to several physical machines to fully guarantee fault tolerance

3

Each server takes a single ring position so the hash ring stays small, simple, and easy to reason

4

Each physical node maps to many ring points, evening out load and avoiding uneven distribution

What are the trade-offs between synchronous and asynchronous replication? If you choose 'Semi-synchronous' replication, what failure scenario are you trying to protect against?

Select the correct answer

1

It waits for all of the replicas to acknowledge, protecting against a network partition splitting the cluster

2

It sends writes to no replicas until much later, protecting against slow disks on the backup follower nodes

3

It waits for at least one replica to acknowledge, protecting against data loss if the primary crashes

4

It writes only to the primary and defers replicas, protecting against read overload on the follower nodes

Explain the difference between Leader-based, Multi-leader, and Leaderless replication.

Select the correct answer

1

Leader-based allows writes anywhere; multi-leader forbids writes entirely; leaderless routes all writes through one fixed node

2

Leader-based sends writes to one node; multi-leader accepts writes at several needing conflict resolution; leaderless uses quorums

3

Leader-based uses quorum writes; multi-leader routes reads through one node; leaderless never has to resolve any conflicts

4

Leader-based and multi-leader both funnel writes to a single node; leaderless replicates only reads by way of quorum voting

What is the difference between Range Partitioning and Hash Partitioning, and what are the tradeoffs regarding range queries vs. load balancing?

Select the correct answer

1

Both range and hash partitioning preserve key order equally well, so the only real difference lies in how they handle node failures.

2

Range partitioning keeps keys sorted for fast range scans but can create hotspots, while hash spreads load evenly yet scatters ranges.

3

Range partitioning always balances load better than hash, while hash is chosen only when strong consistency across shards is required.

4

Hash partitioning keeps keys sorted for fast range scans but can create hotspots, while range spreads load evenly yet scatters ranges.

What is data rebalancing when partitions are added or removed, and what strategies avoid moving too much data at once?

Select the correct answer

1

Rebalancing pauses the cluster to redistribute data evenly; assigning one giant partition per node lets the system move data without any downtime.

2

Rebalancing rewrites every key's hash when nodes change; hashing modulo the node count keeps movement minimal and avoids remapping most of the keys.

3

Rebalancing shifts partitions when nodes join or leave; using fixed partitions or consistent hashing moves only a small fraction rather than everything.

4

Rebalancing duplicates all partitions onto new nodes on any change; snapshotting the whole dataset first ensures only recent writes need to move.

How does a request get routed to the correct partition or shard, and what are the trade-offs between different routing approaches?

Select the correct answer

1

A DNS lookup alone decides the target partition for every key; this keeps clients simple but requires rehashing the entire keyspace whenever nodes change.

2

Requests are always broadcast to every partition which then ignores irrelevant keys; this avoids routing metadata but wastes bandwidth on each request.

3

A client, a routing proxy, or any contacted node can map keys to partitions; each choice trades where routing logic lives against extra hops and coupling.

4

Only a central coordinator can map keys to partitions; this removes all extra hops but tightly couples clients and forces them to cache the full topology.

What is a Hotspot or Hot Key in a sharded system, and how can you detect and mitigate it at the architectural level?

Select the correct answer

1

A hot key is a shard that has crashed under load; detect via heartbeat timeouts and mitigate by promoting a follower replica to take over writes.

2

A hot key is one key or narrow range getting outsized traffic; detect with per-key metrics and mitigate by splitting, salting, or caching it.

3

A hot key is a key with an expired lease; detect through version vectors and mitigate by rerouting all traffic to a dedicated strongly consistent leader.

4

A hot key is a key that is never accessed; detect with per-key metrics and mitigate by merging cold shards together and disabling replication for it.

What is chain replication, and what are its trade-offs compared to leader-based replication?

Select the correct answer

1

Writes pass head to tail along a chain and reads come from the tail, giving strong consistency but adding latency and slower recovery on failures.

2

Writes and reads both hit the middle node of the chain, balancing load evenly but sacrificing durability whenever any single node in the chain fails.

3

Writes go to any node and propagate backward to the head, giving eventual consistency with very low latency but frequent conflicts needing reconciliation.

4

Writes are broadcast to all nodes at once and reads use quorum voting, maximizing availability but reducing overall write throughput under heavy load.

What is anti-entropy, and how do read-repair and Merkle trees help replicas converge in a leaderless system?

Select the correct answer

1

Anti-entropy is a compression scheme for logs; read-repair discards old versions during reads and Merkle trees compact the write-ahead log into snapshots.

2

Anti-entropy is a locking protocol between replicas; read-repair blocks writes during reads and Merkle trees enforce a total ordering of all updates.

3

Anti-entropy is a background sync of replicas; read-repair updates stale copies during reads and Merkle trees pinpoint diverging ranges cheaply.

4

Anti-entropy is a leader election method; read-repair chooses a new coordinator on reads and Merkle trees track which node holds the write lease now.

What are sloppy quorums and hinted handoff, and how do they improve availability in Dynamo-style systems?

Select the correct answer

1

Sloppy quorum requires a strict majority of home nodes for writes and hinted handoff replays a full log to rebuild any node that fell permanently behind.

2

Sloppy quorum accepts writes on any reachable nodes during failures and hinted handoff holds them temporarily to deliver back to the home node later.

3

Sloppy quorum elects a temporary leader to serialize writes and hinted handoff caches responses at the coordinator to speed up repeated key lookups.

4

Sloppy quorum blocks all writes until every home replica responds and hinted handoff queues reads so clients always see the most recently committed value.

What is a Lamport Clock, and how does it differ from a Vector Clock? What specific problem do Vector Clocks solve that Lamport Clocks cannot?

Select the correct answer

1

Lamport clocks need synchronized physical clocks whereas vector clocks do not need any.

2

Vector clocks can detect whether two events are concurrent, which Lamport clocks cannot.

3

Vector clocks reduce message overhead by storing just a single integer per node event.

4

Lamport clocks guarantee a total event ordering while vector clocks give only partial order.

Explain the 'Happens-Before' relationship. How do we determine if two events are concurrent?

Select the correct answer

1

Two events are concurrent if one causally depends on the other through a message.

2

Two events are concurrent if they occur at the same physical wall-clock time value.

3

Two events are concurrent if neither one happens-before the other in the ordering.

4

Two events are concurrent if they happen to share the same Lamport timestamp number.

Why can't we rely on physical system clocks (NTP) for ordering events in a distributed system? What is 'clock skew' and how does it affect distributed logs?

Select the correct answer

1

Clock skew only affects a single local machine and never the event ordering across nodes.

2

Physical clocks are strictly monotonic, so under load they simply overcount logged events.

3

Physical clocks drift and differ between nodes, so their timestamps cannot reliably order events.

4

NTP synchronizes clocks perfectly, but network latency alone reorders the log timestamps.

What is a Lamport Clock, and how does it allow us to determine the partial ordering of events without a synchronized physical clock?

Select the correct answer

1

It synchronizes all node counters to a single global value that is shared cluster-wide.

2

It assigns counters so that equal timestamps always imply the two events are concurrent.

3

It records the physical wall-clock time and rounds it down to the nearest logical tick.

4

It assigns counters so that if a happens-before b, then a's timestamp is less than b's.

What is Last-Write-Wins (LWW), and why is it dangerous in systems where physical clocks are not perfectly synchronized?

Select the correct answer

1

It requires a central coordinator that becomes a bottleneck under any heavy write load.

2

It always keeps both conflicting values, forcing the application to merge them by hand.

3

A skewed clock can give an older write a higher timestamp, so it overwrites a newer write.

4

It orders writes by arrival at the server, silently dropping the update that came first.

Explain the concept of 'Clock Skew' and its impact on distributed transaction ordering.

Select the correct answer

1

It is the gradual slowing of one clock over time, which duplicates transaction identifiers.

2

It is the delay of network packets, which forces transactions to be retried repeatedly.

3

It is the drift of a single clock, which affects only local reads and never global commits.

4

It is the difference between clocks on nodes, which can misorder timestamp-based transactions.

Explain how a Lamport Clock works. If two events have the same Lamport timestamp, can we determine which one happened first? Why or why not?

Select the correct answer

1

A counter increments per event and travels with messages; equal timestamps mean the events occurred simultaneously in real wall-clock time.

2

A counter increments per event and travels with messages; equal timestamps mean one event definitely happened-before the other one.

3

A counter increments per event and travels with messages; equal timestamps mean the events are concurrent and cannot be ordered.

4

A counter increments per event and travels with messages; equal timestamps let us break ties using each node's unique process identifier.

How do Vector Clocks allow us to detect 'causal' relationships between events? In what scenario would a Vector Clock show that two writes are 'concurrent' or 'conflicting'?

Select the correct answer

1

Each node tracks a per-node counter vector; two writes are concurrent when neither vector is fully greater-or-equal to the other.

2

Each node tracks a per-node counter vector; two writes are concurrent when the summed totals of both vectors happen to match.

3

Each node tracks a per-node counter vector; two writes conflict only when both vectors are exactly identical in every position.

4

Each node tracks a per-node counter vector; two writes conflict when one vector is strictly greater than the other one.

What are Hybrid Logical Clocks (HLC), and why are they used in modern databases like CockroachDB or Spanner?

Select the correct answer

1

They synchronize every node's clock via NTP so that logical counters are no longer needed.

2

They replace physical clocks entirely with vector clocks to guarantee a total global ordering.

3

They use GPS and atomic clocks to remove all uncertainty from the timestamps across nodes.

4

They combine physical time with a logical counter to preserve causality with near-real-time values.

What is Clock Skew and Clock Drift, and how do these physical realities impact distributed algorithms like Spanner's TrueTime?

Select the correct answer

1

Skew is the rate of divergence, drift is the difference; TrueTime avoids both by using plain vector clocks.

2

Skew is the difference between clocks, drift is how fast they diverge; TrueTime bounds and waits out this uncertainty.

3

Skew and drift are the very same measure; TrueTime eliminates them completely using only atomic clocks.

4

Skew is network latency, drift is packet loss; TrueTime corrects them by retrying commits until synced.

What is TrueTime in Google Spanner, and how does bounding clock uncertainty enable externally consistent transactions?

Select the correct answer

1

An API returning a logical Lamport counter; Spanner increments it per transaction to totally order commits without physical clocks.

2

An API returning a time interval with bounded error; Spanner waits out that uncertainty before committing to guarantee correct ordering.

3

An API returning a single exact timestamp from atomic clocks; Spanner uses it to skip locking and commit transactions instantly.

4

An API returning a time interval with bounded error; Spanner picks the earliest bound and commits immediately without any waiting.

What is 'Replication Lag,' and what specific anomalies can a user see because of it? How would you design a system to prevent a user from seeing their own update disappear after a refresh?

Select the correct answer

1

Delay between leader and replica updates; a user may miss their own write, fixed by caching every read response indefinitely on the client side.

2

Delay between leader and replica updates; a user may miss their own write, fixed by disabling all replicas and reading only the leader always.

3

Delay between leader and replica updates; a user may miss their own write, fixed by routing reads to an up-to-date replica after writing.

4

Delay between leader and replica updates; a user may miss their own write, fixed by adding more replicas so lag is spread across many nodes.

What are the practical risks of Eventual Consistency? How do concepts like 'Read-your-writes' or 'Monotonic Reads' help mitigate the poor user experience of eventual consistency?

Select the correct answer

1

Stale reads are the risk; read-your-writes ensures you see others' updates and monotonic reads prevents two reads returning same value.

2

Stale reads are the risk; read-your-writes ensures you see your own updates and monotonic reads prevents reads going backward in time.

3

Stale reads are the risk; read-your-writes replicates data synchronously and monotonic reads discards older versions of data permanently.

4

Stale reads are the risk; read-your-writes forces every node to lock and monotonic reads guarantees strict global ordering of all writes.

What are the trade-offs of Eventual Consistency, and how does a system eventually converge if multiple nodes receive conflicting updates?

Select the correct answer

1

It trades consistency for availability and low latency; conflicting updates converge via resolution rules like last-write-wins or CRDTs.

2

It trades partition tolerance for consistency; conflicting updates converge by pausing the system until an administrator resolves them.

3

It trades availability for strong consistency; conflicting updates converge by rejecting all but the first write that any node received.

4

It trades latency for durability guarantees; conflicting updates converge by requiring a quorum to vote on every single write operation.

What are CRDTs (Conflict-free Replicated Data Types), and how do they allow for concurrent updates without a central coordinator?

Select the correct answer

1

Data types whose merge relies on last-writer-wins timestamps only, so replicas discard all but the newest concurrent update always.

2

Data types that use a central lock manager to serialize merges, so replicas converge to the same state regardless of update order.

3

Data types that broadcast every operation through a leader node, so replicas apply updates in a single globally agreed order.

4

Data types whose merge is commutative, associative and idempotent, so replicas converge to the same state regardless of update order.

Explain the high-level roles in the Raft consensus algorithm (Leader, Follower, Candidate). How does the system ensure that two leaders are never elected for the same term?

Select the correct answer

1

Followers elect one Candidate per term; a candidate needs a majority of votes, and each node votes once, so two leaders can't win a term.

2

Followers elect one Candidate per term; a candidate needs unanimous votes from all nodes, each voting once, so two leaders can't win a term.

3

Followers elect one Candidate per term; a candidate needs the highest node ID, and ties break randomly, so two leaders can't win a term.

4

Followers elect one Candidate per term; a candidate needs approval from the previous leader, which steps down, so two leaders can't win.

What is 'Leader Election', and what happens to a distributed system when the leader becomes partitioned from the followers?

Select the correct answer

1

Choosing one node to coordinate; if it is partitioned, both sides keep their leaders and freely commit writes independently.

2

Choosing one node to coordinate; if it is partitioned, followers time out and elect a new leader while the old one is isolated.

3

Choosing one node to coordinate; if it is partitioned, the whole cluster halts until the original leader rejoins the network.

4

Choosing one node to coordinate; if it is partitioned, followers promote themselves so every node becomes its own leader.

How do heartbeats and election timeouts work together in a consensus protocol? What happens if the election timeout is set too low or too high?

Select the correct answer

1

Heartbeats sync clocks between nodes; too low drifts the clocks apart, too high causes two duplicate leaders to be elected at once.

2

Leaders send heartbeats to reset follower timers; too low causes needless elections, too high delays detecting a failed leader.

3

Followers send heartbeats to the leader to reset timers; too low wastes bandwidth, too high blocks new clients from connecting.

4

Heartbeats commit log entries while timeouts trigger snapshots; too low corrupts the log, too high exhausts available disk space.

What are the different message ordering guarantees, FIFO, causal, and total order, and when does each matter?

Select the correct answer

1

FIFO orders all messages globally, causal orders per sender, total respects causality only.

2

FIFO and causal both give one global order, while total order only tracks per-sender order.

3

FIFO preserves per-sender order, causal respects happens-before, total gives one global order.

4

FIFO respects happens-before, causal preserves per-sender order, total relies on timestamps.

What is the FLP Impossibility Result, and why is it impossible to guarantee consensus in a purely asynchronous system if even one node can fail?

Select the correct answer

1

Consensus fails because network partitions permanently split nodes into groups that can never reconcile their divergent states.

2

No protocol reaches consensus because Byzantine nodes send conflicting messages that end up corrupting every honest node's decision.

3

Consensus is impossible because asynchronous clocks drift apart and nodes can never agree on a shared global ordering of events.

4

No deterministic protocol can guarantee consensus when a node may crash, since a slow node is indistinguishable from a failed one.

What is the difference between Paxos and Raft, and why is Raft often preferred for practical implementations despite Paxos being the theoretical gold standard?

Select the correct answer

1

Raft avoids the need for a leader entirely, letting all nodes propose values symmetrically, which removes the single point of failure.

2

Raft requires fewer nodes to reach a quorum than Paxos does, so Raft achieves consensus faster under the same failure conditions.

3

Raft tolerates Byzantine failures while Paxos only handles crash faults, so Raft is safer for adversarial production environments.

4

Raft decomposes consensus into clear subproblems like leader election and log replication, making it easier to implement correctly.

What is the role of a 'Term' or 'Epoch' in consensus protocols like Raft or Paxos?

Select the correct answer

1

A unique identifier assigned to each client request so that duplicate commands are never applied more than once.

2

A counter of committed log entries that is used to detect when a follower has fallen behind the current leader's state.

3

A fixed time interval after which every node must restart and re-synchronize its entire log with the rest of the cluster.

4

A monotonically increasing number that orders leadership periods and lets nodes reject stale messages from older leaders.

In the context of distributed algorithms, what is the difference between a 'safety' property and a 'liveness' property? Why is it often impossible to guarantee both in an asynchronous system (FLP impossibility)?

Select the correct answer

1

Safety means nothing bad happens; liveness means something good eventually happens; FLP shows you can't guarantee both asynchronously.

2

Safety means messages arrive in order; liveness means they arrive quickly; FLP shows ordering and speed conflict once nodes fail.

3

Safety means data gets replicated; liveness means data is durable; FLP shows replication and durability trade off in async systems.

4

Safety means the system stays available; liveness means it stays consistent; FLP shows these two cannot coexist during a partition.

What is the difference between basic Paxos and Multi-Paxos, and why is Multi-Paxos more practical for replicated state machines?

Select the correct answer

1

Basic Paxos tolerates Byzantine faults per instance; Multi-Paxos drops that guarantee to gain speed on trusted replicas.

2

Basic Paxos agrees on many values at once; Multi-Paxos restricts each round to a single value to strengthen safety guarantees.

3

Basic Paxos needs a leader for every round; Multi-Paxos removes the leader so acceptors decide values independently in parallel.

4

Basic Paxos agrees on one value per instance; Multi-Paxos elects a stable leader to skip prepare phases across many entries.

What is the ZAB protocol used by ZooKeeper, and how does it differ from Paxos and Raft?

Select the correct answer

1

ZAB is a crash-recovery atomic broadcast protocol guaranteeing primary-order delivery, tailored for ZooKeeper's state updates.

2

ZAB is a leaderless voting protocol where clients directly commit, unlike Paxos and Raft which centralize all writes on a leader.

3

ZAB is a Byzantine-tolerant protocol that signs every message, unlike Paxos and Raft which assume only simple crash failures.

4

ZAB is a gossip-based protocol that spreads updates eventually, unlike Paxos and Raft which require strict synchronous quorums.

What is state machine replication, and why is a replicated log central to building consistent distributed systems?

Select the correct answer

1

A leader computes results and pushes final state snapshots down to all of its follower replicas.

2

Replicas execute the same deterministic commands in the same log order, reaching identical states.

3

Each replica logs commands locally and resolves any ordering conflicts using vector clock values.

4

Replicas run independent commands and periodically merge their divergent states into a single one.

What is total order broadcast (atomic broadcast), and why is it considered equivalent to consensus?

Select the correct answer

1

It only guarantees delivery, so it stays strictly weaker than actually solving consensus.

2

It orders messages by timestamps, which fully sidesteps the need for any agreement step.

3

Delivering messages in one agreed identical order requires nodes to agree on that order.

4

It broadcasts messages faster than consensus by avoiding any form of leader coordination.

What is the Chandy-Lamport algorithm for capturing a consistent global snapshot of a distributed system?

Select the correct answer

1

Processes broadcast their states, and the initiator selects the most recent consistent combination.

2

Each process periodically saves its state and messages, then merges them by vector clock timestamps.

3

A process records its state on a marker, then records incoming channel messages until markers arrive.

4

A coordinator freezes all processes simultaneously and copies their states using one global clock.

How does PBFT achieve consensus in the presence of Byzantine nodes, and how many faulty nodes can it tolerate?

Select the correct answer

1

Three-phase pre-prepare/prepare/commit voting tolerates f faulty of 3f+1 nodes.

2

Leader-based log replication tolerates f crashed nodes out of 2f+1 total nodes.

3

Two-phase coordinator voting tolerates f faulty nodes out of 2f+1 total nodes.

4

Proof-of-work mining rounds tolerate any minority fraction of the nodes acting maliciously here.

Why is Idempotency critical in distributed systems, and how would you design an idempotent API?

Select the correct answer

1

Retries can duplicate requests, so an idempotency key lets the server dedupe repeated calls.

2

Networks drop packets, so caching every response forever guarantees identical repeated results.

3

Servers can crash, so wrapping each request in a distributed lock prevents all duplicate effects.

4

Clients may reorder calls, so sequence numbers force the server to process requests in order.

Explain the Two-Phase Commit (2PC) protocol. What is its biggest blocking weakness if the coordinator fails?

Select the correct answer

1

If the coordinator fails during commit, participants automatically elect a new coordinator instantly.

2

If the coordinator fails after prepare, participants keep locks and block until it recovers.

3

If the network partitions, participants each unilaterally abort, causing inconsistent partial commits.

4

If a participant fails during prepare, the coordinator silently commits without its acknowledgment.

What is the Saga pattern, and how does it maintain data consistency without using a distributed lock?

Select the correct answer

1

It holds a global lock across every service so all local transactions commit or roll back together.

2

It splits work into local transactions, each with a compensating action that undoes it on failure.

3

It uses a coordinator that votes with all nodes before permanently committing any local transaction step.

4

It queues all writes centrally and replays them in order so no two services ever conflict at once.

Explain the difference between 'At-least-once', 'At-most-once', and 'Exactly-once' delivery guarantees.

Select the correct answer

1

At-least-once never loses or duplicates; at-most-once can duplicate; exactly-once may lose some messages occasionally.

2

At-least-once always duplicates messages; at-most-once always loses some; exactly-once simply retries until acknowledged.

3

At-least-once can lose but never duplicates; at-most-once can duplicate but never loses; exactly-once does neither.

4

At-least-once can duplicate but never loses; at-most-once can lose but never duplicates; exactly-once does neither.

What is the Transactional Outbox pattern, and how does it solve the problem of atomically updating a database and sending a message to another system?

Select the correct answer

1

It caches the message in memory and retries publishing it until the database transaction is fully committed.

2

It sends the message first and only commits the database write after the broker confirms delivery success.

3

It wraps the database write and the broker publish inside one distributed two-phase commit spanning both.

4

It writes the message to an outbox table in the same transaction, then a relay publishes it separately.

Compare 2PC vs. Sagas. When would you choose one over the other in terms of atomicity vs. availability?

Select the correct answer

1

Both provide identical atomicity, but Sagas need a central coordinator while 2PC does not.

2

Sagas give strong atomicity via locking; 2PC favors availability using compensating actions.

3

2PC gives strong atomicity via locking; Sagas favor availability using compensating actions.

4

2PC avoids locking for higher availability; Sagas block resources to ensure strict atomicity.

What is the difference between 'At-least-once' and 'Exactly-once' delivery? Why is 'Exactly-once' often described as an abstraction rather than a literal network guarantee?

Select the correct answer

1

At-least-once may deliver duplicates; exactly-once layers deduplication and idempotency on top of retries.

2

At-least-once needs no acknowledgements; exactly-once is guaranteed natively by most network transport layers.

3

At-least-once sends once without retries; exactly-once retransmits packets until the receiver finally confirms.

4

At-least-once drops messages on failure; exactly-once uses TCP acknowledgements to guarantee single delivery.

What is the Saga pattern, and how do 'Choreography' and 'Orchestration' compare in the context of long-running distributed transactions?

Select the correct answer

1

Choreography commits every step in one transaction; orchestration splits steps across several independent transactions.

2

Choreography uses a central coordinator issuing commands; orchestration lets services react to events fully independently.

3

Choreography coordinates via events with no central controller; orchestration uses a coordinator directing each step.

4

Choreography relies on a distributed lock per step; orchestration relies on compensating events broadcast to services.

What is the Three-Phase Commit (3PC) protocol, and how does it attempt to solve the blocking problem of 2PC?

Select the correct answer

1

It replaces the coordinator with a quorum vote so a majority of nodes can always commit without blocking.

2

It removes the prepare phase and lets each node commit locally, reconciling any conflicts asynchronously afterward.

3

It adds a pre-commit phase and timeouts so nodes can proceed independently if the coordinator fails.

4

It adds a rollback phase so any node timing out can undo commits and restart the whole protocol.

What is the TCC (Try-Confirm-Cancel) pattern, and how does it differ from a Saga?

Select the correct answer

1

TCC commits every step atomically together; Saga splits steps but keeps a distributed lock held throughout flow.

2

TCC compensates with reversing transactions after commit; Saga reserves resources first and confirms or cancels them.

3

TCC reserves resources in a Try step, then Confirm or Cancel; Saga compensates only after committing.

4

TCC uses a central coordinator with locks; Saga uses events among services with no reservation phase at all.

What is distributed deadlock, and how can it be detected or prevented across multiple nodes?

Select the correct answer

1

Deadlock is detected by counting retries per node; increasing the retry limit eventually resolves any blocked cycle.

2

Cycles in a local commit log indicate deadlock; a central coordinator must restart every blocked node together.

3

Cycles in a global wait-for graph indicate deadlock; timeouts or resource ordering can prevent it.

4

Deadlock occurs only when clocks drift; synchronizing timestamps across all nodes fully prevents it from happening.

Explain the Gossip (Epidemic) Protocol. How does it allow a cluster of thousands of nodes to maintain a membership list without a central registry?

Select the correct answer

1

Each node periodically exchanges state with a few random peers, so updates spread exponentially across the whole cluster.

2

Every node continuously multicasts its full state to all other nodes so the cluster converges instantly on any change.

3

A designated registry node broadcasts membership updates to every peer at fixed intervals to keep all of them synchronized.

4

Nodes elect a leader that keeps the authoritative list and pushes deltas down a spanning tree to all of its followers.

What is the 'Split Brain' problem, and how do quorums and fencing tokens prevent it?

Select the correct answer

1

A memory leak splits a node's heap into stale data; quorums cache the majority value and fencing tokens later purge it out.

2

A partition lets two sides both act as leader; quorums require a majority to act and fencing tokens reject stale writers.

3

A slow disk creates duplicate log entries on a node; quorums deduplicate the writes and fencing tokens compress the logs.

4

A clock skew makes two nodes disagree on time; quorums resynchronize the clocks and fencing tokens timestamp every message.

What is failover, and how does a system detect failure and promote a standby without causing inconsistency?

Select the correct answer

1

A standby continuously mirrors writes and takes over instantly; because replication is synchronous, no inconsistency can arise.

2

A load balancer reroutes traffic on failure automatically; since clients retry idempotently, leader consensus is unnecessary.

3

A standby is promoted once failure is detected; quorums and fencing prevent two active primaries from diverging their state.

4

A standby is promoted on any single missed heartbeat at once; a shared timestamp guarantees the old primary rejects all writes.

What is service discovery, and why is it a coordination problem in dynamic distributed systems?

Select the correct answer

1

It caches DNS records locally on each client node; because addresses stay static, no coordination between services is ever needed.

2

It load-balances requests across fixed backend servers; the registry rarely changes, so simple config files handle it without help.

3

It maps service names to current instance locations; endpoints change constantly, so keeping the registry consistent is coordination.

4

It assigns permanent IP addresses to each microservice; since the locations never move, discovery is purely a naming convention.

What is the coordinator/worker (master/worker) pattern, and what are the risks of having a single coordinator?

Select the correct answer

1

A coordinator only stores results while workers self-assign tasks; its failure merely delays reporting but never stalls processing.

2

Workers coordinate peer-to-peer with no central role at all; the main risk is the high messaging overhead among many workers.

3

A coordinator assigns and tracks work across workers; it is a single point of failure and bottleneck unless made highly available.

4

Workers elect a coordinator each cycle to share load evenly; the main risk is workers duplicating tasks when the network is slow.

Why is a simple 'distributed lock' often dangerous? Explain how 'Leases' and 'Fencing Tokens' solve the problem of a process holding a lock but being paused.

Select the correct answer

1

A lock can be acquired twice by one thread by mistake; leases add reentrancy counting and fencing tokens track the recursion depth safely.

2

A lock server becomes a throughput bottleneck under heavy load; leases batch the requests and fencing tokens shard the lock across partitions.

3

A lock without a timeout blocks forever if the client crashes; leases add retry logic and fencing tokens replay the queued operations later.

4

A paused process may resume after its lease expired and another holds the lock; monotonic fencing tokens let storage reject its stale write.

How do coordination services like ZooKeeper, etcd, and Chubby provide primitives such as locks and configuration, and why not build these yourself?

Select the correct answer

1

They use a single master database with row locking so clients see one truth; building your own just needs one shared SQL table.

2

They rely on client-side timeouts and retries to reach agreement; you avoid them because they add unnecessary network latency.

3

They broadcast state via gossip for eventual convergence; you skip them since any key-value cache gives you the same guarantees.

4

They run a consensus algorithm over a replicated log to offer consistent primitives; rolling your own correctly is extremely hard.

What is the SWIM protocol for membership and failure detection, and how does it improve on naive gossip?

Select the correct answer

1

It replaces gossip with a consensus round so all nodes agree on membership, trading extra latency for much stronger consistency.

2

It uses TCP keepalives between every pair of nodes to detect failures instantly, avoiding the false positives of random probing.

3

It centralizes failure detection in one monitor that heartbeats every node, reducing the message overhead of full gossip broadcasts.

4

It separates failure detection from dissemination, using randomized ping and ping-req probes plus piggybacked membership updates.

What is the difference between 'Batch Processing' and 'Stream Processing' at a conceptual level?

Select the correct answer

1

Batch processes each record the instant it arrives; stream collects records into fixed windows that run only on a nightly schedule for reports.

2

Batch processes bounded, stored datasets in scheduled runs; stream processes unbounded data continuously as records arrive with low latency.

3

Batch runs only on a single machine at a time; stream always distributes its work across many machines to guarantee sub-millisecond latency.

4

Batch keeps results in memory without durability; stream writes every intermediate result to disk first to survive failures in long jobs.

How does load balancing work as a distributed-systems building block, and what are common strategies for distributing requests across nodes?

Select the correct answer

1

A balancer replicates each request to all backend nodes at once using strategies like broadcast fanout, gossip flooding, or leader election.

2

A balancer distributes incoming requests across backend nodes using strategies like round-robin, least-connections, or consistent hashing.

3

A balancer schedules background jobs onto idle backend nodes using strategies like priority queues, fair sharing, or deadline preemption.

4

A balancer merges responses from every backend node into one reply using strategies like quorum voting, majority merge, or weighted averaging.

Explain the 'Thundering Herd' problem. How do jitter and exponential backoff help mitigate it?

Select the correct answer

1

Cache expiry floods the database; backoff shards the cache and jitter replicates hot keys across additional nodes to spread the pressure.

2

Simultaneous client retries overwhelm a resource; backoff lengthens gaps between attempts and jitter randomizes their timing.

3

A single slow client blocks others; backoff caches its responses and jitter deduplicates identical requests before they reach the server.

4

Nodes gossip too frequently; backoff compresses their messages and jitter batches them so total network bandwidth stays within safe limits.

Explain the MapReduce execution model. Why is 'Data Locality' a core principle of its design?

Select the correct answer

1

Mappers process splits in parallel and shuffle by key to reducers; running tasks where data resides avoids costly network transfer of inputs.

2

A master streams records to workers continuously; locality caches intermediate keys in memory so repeated jobs skip re-reading the data blocks.

3

Reducers read all input first then mappers aggregate by key; locality keeps every reducer's output on one node to simplify the final merge.

4

Mappers and reducers run on separate clusters via a queue; locality replicates code to all nodes so identical binaries execute the workload.

Explain 'Backpressure'. Why is it critical for a downstream service to signal its inability to keep up with requests?

Select the correct answer

1

It lets a fast upstream service push extra work downstream, keeping buffers full so the slower consumer never sits idle during traffic bursts.

2

It lets an overloaded downstream service signal upstream to slow down, preventing queue buildup, resource exhaustion, and cascading failure.

3

It duplicates requests across downstream replicas, ensuring at least one copy is processed even when a single consumer node becomes overloaded.

4

It compresses requests before sending them downstream, cutting bandwidth so the consumer can process larger batches within its memory budget.

What is a distributed file system like GFS or HDFS at a conceptual level, and how does it achieve fault tolerance for large data?

Select the correct answer

1

Files are striped across nodes without copies, and lost blocks are recomputed from parity kept on a single dedicated metadata master node.

2

Each client caches the entire file locally and periodically syncs edits, so a node crash is recovered from the caches of connected clients.

3

Large files are split into blocks spread across many machines, each block replicated on multiple nodes so data survives individual failures.

4

Large files are kept whole on one powerful node and mirrored nightly to a backup, so a hardware failure loses at most a day of data.

Why do retries in a distributed system need backoff and jitter, and how can naive retries make an outage worse?

Select the correct answer

1

Naive retries block the client thread forever; backoff runs attempts in parallel and jitter caps the concurrent connections each client may open.

2

Naive retries corrupt in-flight data; backoff rolls back the partial writes and jitter randomizes which write commits first among competing clients.

3

Naive immediate retries multiply load on a struggling service; backoff spaces attempts and jitter desynchronizes clients to avoid retry storms.

4

Naive retries drop the original request; backoff resends it to another region and jitter picks that region randomly to balance total load evenly.

What are 'Hedged Requests' and how do they help reduce tail latency in a distributed environment?

Select the correct answer

1

Split each request across several replicas and combine their partial responses, reducing the amount of work any single node must do alone.

2

Send the same request to a second replica after a short delay and take whichever response returns first, cutting slow tail latencies.

3

Queue requests locally and send them only when a replica reports idle capacity, avoiding overload so no request ever hits a busy node.

4

Retry a failed request repeatedly on the same replica with growing delays until it succeeds, ensuring transient slowdowns never surface.