school-of-sre/courses/big_data/intro.md

School of SRE: Big Data

Pre - Reads

• Basics of Linux File systems.
• Basic understanding of System Design.

Target Audience

The concept of Big Data has been around for years; most organizations now understand that if they capture all the data that streams into their businesses, they can apply analytics and get significant value from it. This training material covers the basics of Big Data(using Hadoop) for beginners, who would like to quickly get started and get their hands dirty in this domain.

What to expect from this training

This course covers the basics of Big Data and how it has evolved to become what it is today. We will take a look at a few realistic scenarios where Big Data would be a perfect fit. An interesting assignment on designing a Big Data system is followed by understanding the architecture of Hadoop and the tooling around it.

What is not covered under this training

Writing programs to draw analytics from data.

TOC:

1. Overview of Big Data
2. Usage of Big Data techniques
3. Evolution of Hadoop
4. Architecture of hadoop
   1. HDFS
   2. Yarn
5. MapReduce framework
6. Other tooling around hadoop
   1. Hive
   2. Pig
   3. Spark
   4. Presto
7. Data Serialisation and storage