Hadoop, (Part 3 of 4): YARN and NiFi

Tackle Hadoop tools and services like NiFi, YARN, and Flume as well as the Spark shell, an alternative to MapReduce. Discover why Hadoop has such a large and growing following among sys admins and data scientists. Learning how Hadoop has something for just about everybody to gain and maintain competitive advantage.

Lesson 1:

• Scalable Computing
• Scalable Computing - YARN
• YARN
• Hadoop = YARN + HDFS
• Managing Data and Processes
• Limitations of MapReduce v1
• YARN Processing.

Lesson 2:

• YARN Daemons
• Resource Manager
• Node Manager
• Job History Server
• Application Master
• Tasks in YARN
• YARN Architecture
• Demo: YARN Tools
• Demo: YARN Parameters.

Lesson 3:

• Hadoop Structure
• Tez
• What is Apache Spark?
• Benefits of Spark vs. MapReduce
• Spark Framework
• Spark Languages.

Lesson 4:

• Demo: The Spark Shell
• Demo: Using Scala.

Lesson 5:

• What Is Flume?
• What Does Flume Do?
• Flume Overview
• Linux ETL
• What is Nifi?
• Working with Nifi
• Nifi Capabilities
• Demo: Linux ETL
• Demo: More Linux Commands.

Lesson 6:

• Demo: Install Nifi
• Demo: install-nifi.sh
• Demo: Run Nifi.

Lesson 7:

• Nifi Components
• The FlowFile
• The GetFile Component
• The UnpackContent Component
• The ControlRate Component
• The EvaluateXPath Component
• The SplitXML Component
• The UpdateAttribute Component
• The AttributesToJSON Component
• The MergeContent Component
• The PutFile Component
• The RouteOnAttribute Component.

Lesson 8:

• Demo: Nifi Workflow
• Demo: Processors
• Demo: Source File.

Lesson 9:

• Demo: GetFile, Unpack Content, and ControlRate
• Demo: Evaluate XPath and Split XML
• Demo: Update Attribute and Parse Records
• Demo: Route on Attribute
• Demo: MergeContent and PutFile
• Demo: Debugging.