Packt+ | Advance your knowledge in tech

0

All Products

Best Sellers

New Releases

Books

Videos

Audiobooks

Free Learning

Hadoop 2.x Administration Cookbook

You're reading from Hadoop 2.x Administration Cookbook Administer and maintain large Apache Hadoop clusters

Product type Paperback

Published in May 2017

Publisher Packt

ISBN-13 9781787126732

Length 348 pages

Edition 1st Edition

Tools

Hadoop

Concepts

System Administration

Author (1):

Aman Singh

View More author details

Table of Contents (20) Chapters

Hadoop 2.x Administration Cookbook

Credits

About the Author

About the Reviewers

www.PacktPub.com

Customer Feedback

Preface

1. Hadoop Architecture and Deployment FREE CHAPTER

2. Maintaining Hadoop Cluster HDFS

3. Maintaining Hadoop Cluster – YARN and MapReduce

4. High Availability

5. Schedulers

6. Backup and Recovery

7. Data Ingestion and Workflow

8. Performance Tuning

9. HBase Administration

10. Cluster Planning

11. Troubleshooting, Diagnostics, and Best Practices

12. Security

Index

Preserving ResourceManager states

It is important to preserve the state of ResourceManager during the restart of RM, so as to keep the application running with minimal interruptions. The concept is that the RM preserves the application state in a store and reloads it on restart. ApplicationMasters (AM) and NodeManagers continuously poll RM for status and re-register with it when available, thus resuming the containers from saved state.

Getting ready

For this recipe, you will again need a running cluster and have completed the previous recipes to make sure the cluster is working fine in terms of HDFS and YARN.

How to do it...

Connect to the master1.cyrus.com master node and switch to user hadoop.
Navigate to the directory /opt/cluster/hadoop/etc/hadoop.
Edit the yarn-site.xml configuration file to make the necessary changes as shown in the following steps.
Enable RM recovery by making changes as shown in the following screenshot:
Specify the state-store to be used for this, as shown in the following...

The rest of the chapter is locked

Register for a free Packt account to unlock a world of extra content!

A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Start free trial

Renews at £13.99/month. Cancel anytime

Authors (1)

Aman Singh

Aman Singh

Gurmukh Singh is a seasoned technology professional with 14+ years of industry experience in infrastructure design, distributed systems, performance optimization, and networks. He has worked in big data domain for the last 5 years and provides consultancy and training on various technologies. He has worked with companies such as HP, JP Morgan, and Yahoo. He has authored Monitoring Hadoop by Packt Publishing

See other products by Aman Singh

Other recommended products

Related to this chapter

Apache Hadoop 3 Quick Start Guide

Apache Hadoop 3 Quick Start Guide

Apache Hadoop is a widely used distributed data platform. It enables large datasets to be efficiently processed instead of using one large computer to store and process the data. This book will get you started with the Hadoop ecosystem, and introduce you to the main technical topics such as MapReduce, YARN and HDFS.

Oct 2018 7h 20m

HBase High Performance Cookbook

HBase High Performance Cookbook

Jan 2017 11h 40m

Mastering Hadoop 3

Mastering Hadoop 3

This is a comprehensive guide to understand advanced concepts of Hadoop ecosystem. You will learn how Hadoop works internally, and build solutions to some of real world use cases. Finally, you will have a solid understanding of how components in the Hadoop ecosystem are effectively integrated to implement a fast and reliable Big Data pipeline

Feb 2019 18h 8m

Apache Hive Essentials

Apache Hive Essentials

Apache Hive helps you deal with data summarization, queries, and analysis for huge amounts of data. This book will give you a background in big data, and familiarize you with your Hive working environment. Next you will cover advanced topics like performance and security in Hive and how to work efficiently to find solutions to big data problems.

Modern Big Data Processing with Hadoop

Modern Big Data Processing with Hadoop

This book presents unique techniques to conquer different Big Data processing and analytics challenges using Hadoop. Practical examples are provided to boost your understanding of Big Data concepts and their implementation. By the end of the book, you will have all the knowledge and skills you need to become a true Big Data expert.

Mar 2018 13h 8m

Mastering Apache Storm

Mastering Apache Storm

With real-world examples and clear explanations, this book will ensure you will have a thorough mastery Apache Storm.You'll get an understanding of deploying Storm on clusters. Introduce yourself to topics such as trident topology, monitoring, Storm Parallelism, scheduler and log processing. Learn how to integrate Storm with other well-known Big Data technologies such as HBase, Redis, Kafka, and Hadoop to realize the full potential of Storm.You will be able to use the knowledge to develop efficient, distributed real-time applications to cater to your business needs.

Aug 2017 9h 28m

Data Lake for Enterprises

Data Lake for Enterprises

The term 'Data Lake' has recently emerged as a prominent term in the big data industry. Data scientists can make use of it in deriving meaningful insights which can be used by businesses to redefine or transform the way they operate. Lambda architecture is also emerging as one of the very eminent patterns in the big data landscape, as it helps to derive useful information from not only the historical data but also correlates real-time data to enable business for taking critical decisions. This book tries to bring these two important aspects into one, namely data lake and lambda architecture.

May 2017 19h 52m

Personalised recommendations for you

Based on your interests and search pattern

Mastering Palo Alto Networks

Mastering Palo Alto Networks

Mastering Palo Alto Networks is a hands-on guide that helps you progress from a beginner to an expert in configuring Palo Alto firewalls. It covers setup, management, and optimization with real-world examples for both on-prem and cloud environments.

May 2025 21h 32m

Cloud Native Anti-Patterns

Cloud Native Anti-Patterns

Uncover and fix cloud native anti-patterns across strategy, culture, security, and operations. This book guides you through the practical steps to avoid common traps and shift your organization toward cloud-native best practices that drive lasting success.

Mar 2025 14h 44m

AWS for System Administrators

AWS for System Administrators

Master the art of designing, deploying, and managing AWS infrastructure with industry best practices. You'll learn to automate workflows, optimize costs, and implement robust disaster recovery strategies for scalable cloud success.

May 2025 14h 12m

Kubernetes for Generative AI Solutions

Kubernetes for Generative AI Solutions

Learn step by step how to design, optimize, and deploy Generative AI projects on Kubernetes. Covering networking, observability, security, scaling, and cost optimization strategies, this guide takes you from first deployment to production excellence.

Jun 2025 11h 8m

Azure Cloud Projects

Azure Cloud Projects

Azure Cloud Projects offers a hands-on introduction to Microsoft Azure, designed to help you build real-world cloud solutions, develop job-ready skills, and apply best practices that are widely used across the cloud computing industry.

May 2025 8h 28m

Sustainable Cloud Development

Sustainable Cloud Development

This book introduces best practices for developing and deploying high-performing, eco-friendly cloud applications, helping you meet sustainability goals while delivering efficient cloud solutions.

Mar 2025 9h 12m

SLIs and SLOs Demystified

SLIs and SLOs Demystified

This comprehensive guide to reliability engineering dives deep into SLIs, SLOs, observability, and incident management. It shows you how to optimize system reliability and make data-driven decisions to balance performance and business goals.

Apr 2025 10h 0m