Apache Spark And Scala Training in Antananarivo

Apache Spark and Scala Training​ Course outline

Module 1: Introduction to Scala

• Introduction to Scala and Development of Scala for Big Data Applications
• Apache Spark

Module 2: Pattern Matching

• Introduction to Pattern Matching
• Uses of Scala
• Concept of REPL (Read Evaluate Print Loop)
• Deep Drive into Scala Pattern Matching
• Type Interface and Higher-Order Function
• Currying and Traits

Module 3: Executing the Scala Code

• Introduction to Scala Interpreter
• Creating Static Members with Companion Objects
• Implicit Classes in Scala
• Different Classes in Scala

Module 4: Classes Concepts in Scala

• Understanding the Constructor Overloading
• Different Abstract Classes
• Hierarchy Types in Scala
• Concept of Object Equality and Val and Var Methods in Scala​

Module 5: Concepts of Traits with Example

• Introduction to Traits in Scala ​
• When to Use Traits?​
• Linearisation of Traits and the Java Equivalent ​
• Boilerplate Code​

Module 6: Scala Java Interoperability and Scala Collection​

• Implementation of Traits in Scala and Java​
• Handling of Multiple Traits Extending​
• Introduction to Scala Collections​
• Classification of Collections ​
• Difference Between Iterator and Iterable in Scale
• List and Sequence in Scala

Module 7: Mutable Collections vs Immutable Collections

• Types of Collections in Scala
• Lists and Arrays in Scala
• List Buffer and Array Buffer
• Queue in Scala
• Stacks and Sets
• Maps and Tuples in Scala

Module 8: Introduction to Spark

• What are Spark and Spark Stack?
• Ways to Resolve Hadoop Drawbacks
• Interactive Operations on Map Reduce
• Spark Hadoop YARN
• HDFS and YARN Revision
• How it is Better Hadoop?
• Deploying Spark Without Hadoop
• Spark History Server
• Cloudera Distribution

Module 9: Spark Basics

• Spark Installation
• Memory Management
• Concept of Resilient Distributed Datasets (RDD)​
• Functional Programming in Spark​

Module 10: Working with RDDs in Spark​

• Creating RDDs ​
• Operations and Transformation in RDD ​
• RDD Partitioning ​
• FlatMap Method ​
• Scala Map Count ​
• Saveastextfiles
• Pair RDD Functions

Module 11: Aggregating Data with Pair RDDs ​

• Introduction to Key-Value Pair in RDDs ​
• How Spark Makes Map-Reduce Operations Faster?​

Module 12: Writing and Deploying Spark Applications​

• Difference Between Spark and Scala
• Set and Set Operations
• List and Tuple
• Concatenating List
• Install Apache Maven

Module 13: Parallel Processing

• Spark Parallel Processing
• Setup Spark Master Code
• Introduction to Spark Partitions
• Data Locality in Hadoop
• Comparing Repartition and Coalesce
• Actions of Spark

Module 14: Spark RDD Persistence

• Execution Flow in Spark
• RDD Persistence Overview
• Spark Terminology
• Distribution Shared Memory vs RDD
• ReduceByKey and SortByKey and AggregateByKey

Module 15: Spark Streaming and Mila

• Introduction to Spark Streaming
• What is Spark Streaming?
• Aspects of Spark Streaming
• How does Spark Streaming Work?
• Broadcast Variables
• Accumulator

Module 16: Spark Variables and RDD Operations

• Variables in Spark
• Numeric RDD Operations

Module 17: Scheduling or Partitioning

• Partitioning in Spark
• Hash Partition and Range Partition
• Scheduling within and Around Applications
• Map Partition with Index
• GroupByKey
• Spark Master High Availability
• Standby Masters with Zookeeper