Fundamentals & Use Cases
Why is Kafka so popular, and what are the key benefits of Apache Kafka over other traditional techniques?
What is Apache Kafka, and what are its primary use cases?
Why is Kafka preferred to REST for certain use cases?
Can you explain 3 to 5 important features of Kafka?
Can Kafka be used for real-time data processing?
What are the origins of Kafka at LinkedIn, and what problem was it originally built to solve?
Why is Kafka used in microservices architecture?
How has Kafka usage changed by 2026?
What is Change Data Capture (CDC) in the context of Kafka?
How does Kafka compare to Amazon SQS/SNS, and when would you choose one over the other?
How does Kafka fit into an event sourcing architecture as the source of truth log?
Topics & Partitions
What is a Kafka topic, and how is it structured?
How does partitioning improve scalability?
What is a topic in Kafka, and how does it differ from a partition?
What is a Kafka partition key?
Can partition count be increased?
What is the role of partitions in Kafka, and how do they affect scalability, ordering, and parallelism?
How does Kafka guarantee message ordering?
Can partition count be decreased?
Why is there no global ordering guarantee across partitions in Kafka, only per-partition ordering?
How would you choose the optimal number of partitions for a Kafka topic, and what are the trade-offs of increasing or decreasing partitions?
Producers
What is a Kafka producer?
How does a Kafka Producer determine which partition to send a message to, and what are the different partitioning strategies?
What are the most important configurations for Kafka producers?
How do Kafka producers achieve fault tolerance?
What is the sticky partitioner, and how does it differ from the default key-hash and round-robin partitioning?
Consumers & Consumer Groups
What is a Kafka consumer?
What happens if there are more consumers than partitions?
What is a Consumer Group, and how does it enable parallel processing and scalability?
Explain the difference between Kafka's high-level and low-level consumer APIs.
Explain the consumer pull model in Kafka.
What is the difference between using subscribe() and assign() in a Kafka consumer?
What strategies can be used for Kafka consumer group management?
What does the consumer isolation.level (read_committed vs read_uncommitted) control?
Offsets Records & Log Internals
What is an Offset in Kafka?
What information does a Kafka record contain beyond key and value, and what are headers used for?
What is consumer offset, and how is it managed in Kafka?
What is the difference between CreateTime and LogAppendTime for a Kafka record's timestamp?
What is the difference between the log-end-offset and the high watermark?
What is the __consumer_offsets topic, and how does Kafka use it to store consumer offsets?
What is the Last Stable Offset (LSO), and how does it relate to transactions and consumer visibility?
What are the .index and .timeindex files in a Kafka log segment, and what purpose do they serve?
Architecture & Cluster Components
What is the role of a Kafka broker?
What are bootstrap servers, and how does a client discover the rest of the Kafka cluster?
What are the core components of Kafka, and how do they interact?
What happens if a Kafka broker goes down?
What is the fundamental architecture of Apache Kafka and how does it differ from traditional message brokers?
What is the Kafka Controller, and what are its responsibilities?
What is preferred leader election, and why does Kafka perform automatic leader rebalancing?
Kafka Connect
Explain the difference between Source and Sink connectors.
What serialization formats are supported by Kafka Connect?
What is Kafka Connect, why was it introduced, and how does it differ from writing custom producers/consumers?
How does Kafka Connect manage offsets for connectors?
What is the role of converters in Kafka Connect?
What are Single Message Transforms (SMTs), and when would you use them?
What is a connector, task, and worker in Kafka Connect?
What is a Dead Letter Queue (DLQ) in Kafka Connect, and when would you use it?
What are Kafka connectors, and how are they used to integrate with external systems?
What are Kafka Connect and Kafka Streams?
Why does Kafka Connect run as a separate service?
Can Kafka Connect produce duplicates?
What is the difference between JsonConverter and AvroConverter in Kafka Connect?
Can Kafka Connect work without Schema Registry?
What are tombstone records used for in Kafka Connect?
What is the difference between standalone and distributed mode in Kafka Connect?
Does Kafka Connect support exactly-once semantics?
What delivery guarantees does Kafka Connect provide?
How does Kafka Connect avoid data loss?
How do you ensure idempotency in sinks with Kafka Connect?
When should you not use SMTs?
How does schema evolution work with Kafka Connect?
Change Data Capture
What database logs does CDC read?
How are updates represented in CDC?
How are deletes handled in CDC?
How does Debezium work with Kafka Connect?
Replication & Durability
What is replication factor?
How does Kafka ensure fault tolerance and data durability?
Explain the role of Leader and Follower replicas in a Kafka partition.
Explain the replication mechanism in Kafka.
How can you increase the replication factor in Kafka?
What are under-replicated and offline partitions, and what do they indicate about cluster health?
What is the In-Sync Replica (ISR) set, and why is it important for durability and availability?
What is the High Watermark, and how does it relate to message visibility and durability?
What is a replication tool in Kafka and what are some of the replication tools available?
What is min.insync.replicas, and how does it interact with acks=all to guarantee durability?
What is unclean leader election, and what is the trade-off between availability and durability it represents?
Delivery Semantics & Transactions
What is at-least-once delivery in Kafka?
How do you handle duplicate messages in Kafka?
What is exactly-once semantics in Kafka?
What is the difference between at-least-once and exactly-once semantics in Kafka?
Explain the three message delivery semantics in Kafka (at-most-once, at-least-once, exactly-once) and how Kafka achieves each.
What are Kafka Transactions, and how are they used to achieve exactly-once semantics in read-process-write loops?
Why can duplicates still happen even when Kafka's exactly-once semantics is enabled, especially with external systems?
What is the transaction coordinator, and how does it manage a Kafka transaction's lifecycle?
What is the role of the transactional.id in Kafka transactions, and why is it important for fencing zombie producers?
Producer Config & Idempotence
What is message compression in Kafka, and why is it used?
What is producer acknowledgement (acks)?
Explain the acks parameter in a Kafka Producer and the trade-offs between acks=0, acks=1, and acks=all.
What is an Idempotent Producer, and how does it prevent duplicate messages?
How do Kafka Producers handle message batching and compression, and what are the relevant configurations?
What is the role of batch size and linger.ms in Kafka producer performance?
What does delivery.timeout.ms control in a Kafka producer, and how does it relate to retries?
What is the role of buffer.memory in a Kafka producer, and what happens when it fills up?
How does max.in.flight.requests.per.connection affect message ordering, especially with retries?
How do producer IDs and sequence numbers enable idempotent delivery under the hood?
Consumer Rebalancing & Offset Management
How do consumers manage offsets, and what is the difference between auto-commit and manual commit (sync/async)?
When would you use auto.offset.reset with earliest, latest, or none?
How does max.poll.records affect consumer behavior and rebalancing?
What can you do with the kafka-consumer-groups CLI tool for inspecting or resetting offsets?
What is a consumer rebalance, what triggers it, and why can it be disruptive?
What is static group membership, and when would you use it?
How does Kafka handle backpressure?
What is the difference between eager rebalancing and cooperative rebalancing?
What is the difference between session.timeout.ms and max.poll.interval.ms, and how do heartbeats work in a consumer?
Compare the range, round-robin, sticky, and cooperative-sticky partition assignment strategies.
Monitoring & Operations
What is Consumer Lag, and how can you monitor and remediate it?
What are some key metrics you would monitor in a Kafka cluster (brokers, producers, consumers)?
You observe high consumer lag for a topic — how would you investigate and troubleshoot this issue?
What is the Kafka AdminClient, and what kinds of operations can you perform with it?
How would you monitor and troubleshoot Kafka performance issues in production?
How do you perform a rolling restart of a Kafka cluster?
What are the most common Kafka production mistakes?
What are quotas in Kafka, and how does throttling protect a cluster from misbehaving clients?
How do you approach sizing and capacity planning for a Kafka cluster?
Schema Registry & Evolution
What is the Schema Registry, and why is it important?
What are subjects and versions in the Schema Registry, and how are they related to topics?
Explain the different schema compatibility modes (backward, forward, full, none) and their implications for schema evolution.
How does the Schema Registry wire format work, and what is the role of the embedded schema id in each message?
Storage Retention & Compaction
What are Kafka logs, and how are they managed?
Describe Kafka's log compaction feature and its typical use cases.
How does Kafka handle message retention and deletion?
What is the difference between cleanup.policy=delete and cleanup.policy=compact?
What is a tombstone record in a compacted topic, and how does it trigger deletion?
How does Kafka store data on disk, and what are log segments?
What is tiered storage in Kafka, and what problem does it solve?
Zookeeper & Kraft
Can Kafka be used without ZooKeeper?
Explain the transition from ZooKeeper to KRaft mode in Kafka. What problems does KRaft solve?
Can you explain ZooKeeper's role in Kafka's architecture, and how Kafka will operate without it in the future?
In KRaft mode, what is the metadata log, and what is the difference between combined and isolated controller modes?
Security
What are SSL/TLS and SASL in Kafka security?
How do you secure Kafka, including encryption (SSL/TLS), authentication (SASL), and authorization (ACLs)?
What Kafka security mechanisms would you implement to ensure that only authorized producers can publish to a topic?
Compare the different SASL mechanisms Kafka supports (PLAIN, SCRAM, GSSAPI/Kerberos, OAUTHBEARER).
How do Kafka ACLs and the authorizer work to control access to topics and operations?
Comparisons With Other Systems
How does Kafka differ from traditional message brokers like RabbitMQ?
What are the main differences between Apache Kafka and Apache Pulsar?
Explain the differences between Kafka and Pulsar in terms of the separation of storage and processing layers and the benefits this provides in Pulsar.
Kafka Streams & Ksqldb
What is a stream in the context of Kafka Streams?
What are common use cases for Kafka Streams?
What is Kafka Streams, and how does it differ from traditional message processing?
Explain the concept of windowing in Kafka Streams (tumbling, hopping, sliding, session windows).
What is Kafka Streams, and what are its key abstractions (KStream, KTable, GlobalKTable)?
How does Kafka Streams differ from the Kafka Consumer API?
What is a topology in Kafka Streams?
What is a KTable in Kafka Streams, and how does it differ from a KStream?
What is the role of stateful and stateless transformations in Kafka Streams?
How does Kafka Streams scale horizontally?
What is the significance of the application ID in Kafka Streams?
What is ksqlDB, and how does it relate to Kafka Streams?
How does Kafka Streams achieve exactly-once processing?
What is the Processor API and when would you use it instead of the DSL in Kafka Streams?
How do state stores work in Kafka Streams, and what is their relationship with changelog topics?
How does Kafka Streams handle fault tolerance?
What is a stream thread and how does it relate to tasks in Kafka Streams?
What are standby replicas in Kafka Streams?
What are common causes of high consumer lag in a Kafka Streams application?
Why prefer event time over processing time for aggregations in Kafka Streams?
What are interactive queries in Kafka Streams, and how do they let you query state stores?
What is the difference between a GlobalKTable and a KTable, and when would you use each in joins?
Performance & Tuning
What are the key Kafka performance tuning parameters?
How does Kafka use the OS page cache and sequential disk I/O to achieve performance?
Compare the trade-offs of the different compression codecs (gzip, snappy, lz4, zstd) in Kafka.
What is zero-copy, and how does it impact Kafka's performance?
How does Kafka ensure high throughput and low latency?
How would you optimize Kafka for high throughput?
High Availability & Disaster Recovery
How can you replay or reprocess historical data in Kafka using seek and offset reset?
How do you ensure message ordering when replaying historical Kafka events?
Explain the concept of rack awareness in Kafka.
What is MirrorMaker 2, and when would you use it for cross-cluster replication?
How would you configure Kafka for high availability?
How would you approach disaster recovery and multi-datacenter deployment with Kafka?