Lecture

Data formats and data wrangling with Hadoop

Related lectures (32)

Covers Spark Data Frames, distributed collections of data organized into named columns, and the benefits of using them over RDDs.

Data Warehouses and Decision Support Systems

Explores data warehouses, decision support systems, OLAP, data lakes, multidimensional data models, and query optimizations.

Advanced Spark Optimization Techniques: Managing Big Data

Discusses advanced Spark optimization techniques for managing big data efficiently, focusing on parallelization, shuffle operations, and memory management.

Handling Data: Intro to Pandas

Introduces the fundamentals of handling data, emphasizing the importance of Pandas and data modeling for effective analysis.

Relational Model: Basics

Introduces the relational model, SQL, keys, integrity constraints, ER translation, weak entities, ISA hierarchies, and SQL vs. noSQL.

Introduction to Database Systems

Covers the fundamentals of database systems, including data modeling, information processing, and the challenges of managing large volumes of data.

Data Warehouses: Introduction and Challenges

Covers the introduction and challenges of data warehouses, including integrating data, managing metadata, and optimizing query performance.

Integrating Scalable Data Storage and Map Reduce Processing with Hadoop

Covers the integration of scalable data storage and map reduce processing using Hadoop, including HDFS, Hive, Parquet, ORC, Spark, and HBase.

Data Wrangling: Transforming Data for Analysis

Covers the process of data wrangling, focusing on transforming and preparing data for analysis.

Knowledge Representation: Semantics and Data Structures

Explores knowledge representation, data structures, semantics, and the challenges of searching for data on the web.

Data Cleaning Challenges: Optimizing Error Detection

Addresses challenges in data cleaning for analysis, proposing optimizations to reduce processing time.

Data: Clouds and Things

Covers the fundamentals of data, cloud storage, and the Internet of Things.