Lecture

Advanced Spark Optimization Techniques: Managing Big Data

Related lectures (32)

Introduces the basics of data science, covering decision trees, machine learning advancements, and deep reinforcement learning.

Big Data Ecosystems: Technologies and Challenges

Covers the fundamentals of big data ecosystems, focusing on technologies, challenges, and practical exercises with Hadoop's HDFS.

Real-time Intelligence: Data Challenges and Hardware Evolution

Explores data challenges and hardware evolution for real-time intelligence in the era of big data.

Data Science: Python for Engineers - Part II

Explores data wrangling, numerical data handling, and scientific visualization using Python for engineers.

Handling Data: Intro to Pandas

Introduces the fundamentals of handling data, emphasizing the importance of Pandas and data modeling for effective analysis.

Collaborative Data Science: Tools and Techniques

Introduces collaborative data science tools like Git and Docker, emphasizing teamwork and practical exercises for effective learning.

Big Data Challenges: Scaling to Massive Data

Explores challenges of handling massive data in the era of big data, discussing solutions like MapReduce and Spark.

Data Modeling: Concepts and Applications

Explores data modeling concepts, SQL implementations, and practical applications in handling missing data.

Data Stream Processing: Apache Kafka and Spark

Covers data stream processing with Apache Kafka and Spark, including event time vs processing time, stream processing operations, and stream-stream joins.

Spark DataFrames: Basics and Optimization

Covers the basics of Spark DataFrames, their advantages, performance comparison with RDDs, and practical demos.

Analytics on Data at Rest and Data in Motion

Explores combining data at rest with data in motion, emphasizing the Lambda architecture complexities and quality assessment of streams and batches.

Water Consumption in Geneva

Explores water consumption data in Geneva, including charts on consumption and losses, available datasets, and data processing phases.