Big Data Technologies
Distributed data processing, Apache Spark RDDs and DataFrames, Apache Kafka event streaming, HDFS storage, and Flink stream processing.
HDFS NameNode/DataNode architecture, block replication, rack awareness, and S3 consistency.
Distributed Storage - HDFS & Object Stores
HDFS NameNode/DataNode architecture, block replication, rack awareness, and S3 consistency.
Resilient Distributed Datasets (RDDs), Spark DataFrames, Catalyst query optimizer, and Spark UI profiling.
Distributed Processing - Apache Spark Engine
Resilient Distributed Datasets (RDDs), Spark DataFrames, Catalyst query optimizer, and Spark UI profiling.
Kafka topics, partitions, consumer groups, log retention policies, and exactly-once semantics.
Real-Time Event Streaming with Apache Kafka
Kafka topics, partitions, consumer groups, log retention policies, and exactly-once semantics.
Micro-batching vs event-driven stream processing, event time windowing, and Flink stateful operations.
Stream Processing Engines - Spark Streaming & Flink
Micro-batching vs event-driven stream processing, event time windowing, and Flink stateful operations.