---
title: "Hadoop Technologies Explained: From MapReduce to HDFS"
description: "Hadoop is a big platform to solve your big data problems. Explained well about hadoop ecosystem architecture and its technologies currently being used across the world."
url: https://srinimf.com/2017/08/22/all-about-hadoop-eco-system-and-its-popular-technologies/
date: 2017-08-22
modified: 2026-10-10
author: "Srini"
image: https://i0.wp.com/srinimf.com/wp-content/uploads/2017/08/pexels-photo-2228585.jpeg?fit=1880%2C1253&ssl=1
categories: ["Hadoop"]
tags: ["Hadoop eco system", "Hadoop tips and Tricks"]
type: post
lang: en
---

# Hadoop Technologies Explained: From MapReduce to HDFS

[Hadoop ecosystem](https://srinimf.com/2013/04/16/hadoop-mapreduce-concept/) comes with a set of technologies. An ecosystem of Hadoop would solve [big data](https://srinimf.com/2013/06/30/hadoop-and-big-data/) problems.

Related[4 Bigdata Patterns and Databases Everyone Should Know](https://srinimf.com/2018/08/20/store-your-bigdata-these-are-4-popular-databases-you-need-to-know/)

Related[Hadoop Integration With Mainframe Here is Useful Story](https://srinimf.com/2013/10/04/mainframe-and-hadoop/)

Related[Sweet Spots of DB2 V11 with Hadoop](https://srinimf.com/2015/06/05/sweet-spots-of-db2-v11-with-hadoop/)

Related[Sqoop, Flume and Storm Understand The Differences Quickly](https://srinimf.com/2019/04/21/hadoop-sqoop-vs-flume-vs-storm-to-process-data/)

Related[Technologies We Could Live Without](https://srinimf.com/2024/11/29/technologies-we-could-live-without/)

Here is explained nature of the work of each technology in the context of the ecosystem.

Related[Top Data Analyzing Tools: Pig vs Hive vs Jaql for Big Data](https://srinimf.com/2013/08/23/pig-hive-jaql-differences/)

## Hadoop ecosystem technologies

1. HDFS
   1. HDFS is the one, which makes it possible to store different types of large data sets (i.e. structured, unstructured and semi structured data).
2. YARN
   1. The basic functions of YARN is ResourceManager and NodeManager.
3. MAP REDUCE
   1. MapReduce is a software framework which helps in writing applications that processes large data sets using distributed and parallel algorithms inside Hadoop environment.
4. SPARK
   1. In memory data processing
5. HIVE
   1. Facebook created HIVE for people who are fluent with SQL. Thus, HIVE makes them feel at home while working in a Hadoop Ecosystem.
6. PIG
   1. PIG has two parts: **[Pig Latin](https://srinimf.com/2017/09/07/pig-latin-tutorial/)**, the language and **the pig runtime,** for the execution environment. You [can be](https://srinimf.com/2017/09/08/the-story-develop-scala-applications-to-access-hadoop-data/)tter understand it as Java and JVM.
   2. It supports* pig latin* language, which has SQL like command structure.
7. HBASE
   1. NOSQL database
8. MAHOUT
   1. Mahout provides an environment for creating machine learning applications which are scalable
9. SPARK
   1. Apache Spark is a framework for real time data analytics in a distributed computing environment.
10. DRILL
    1. It is a replica of Google Dremel.
    2. It supports different kinds [NoSQL databases](https://srinimf.com/2016/02/06/the-2-best-things-about-nosql-databases/) and [file system](https://srinimf.com/2017/12/30/beyond-hdfs-to-store-massive-hadoop-data/)s, which is a powerful feature of Drill.  For example: Azure Blob Storage, Google Cloud Storage, HBase, MongoDB, MapR-DB HDFS, MapR-FS, Amazon S3, Swift, NAS and local files.
11. ZOOKEEPER
    1. Apache Zookeeper is the coordinator of any [Hadoop job](https://srinimf.com/2017/12/04/hadoop-workflow-process/) which includes a combination of various services in a Hadoop Ecosystem.
12. OOZIE
    1. Consider Apache Oozie as a clock and alarm service inside Hadoop Ecosystem. For Apache jobs, Oozie has been just like a scheduler.
13. FLUME
    1. The Flume is a service which helps in ingesting unstructured and semi-structured data into HDFS.
14. SQOOP
    1. Flume only ingests unstructured data or semi-structured data into HDFS.
    2. While Sqoop can import as well as export structured data from RDBMS or Enterprise data warehouses to HDFS or vice versa.
15. SOLAR & LUCENE
    1. Apache Solr and Apache Lucene are the two services which are used for searching and indexing in Hadoop Ecosystem.
16. AMBARI
    1. Ambari is an Apache Software Foundation Project which aims at making Hadoop ecosystem more manageable.
17. HADOOP
    1. It is the overall architecture of Hadoop system

Related[AWS EMR Tutorial: Beginner’s Guide to Big Data Processing on the Cloud](https://srinimf.com/2025/09/26/aws-emr-tutorial-beginners-guide-to-big-data-processing-on-the-cloud/)

### **3 Top Hadoop Platforms**

1. IBM
2. Cloudera
3. AWS

References

- [10 Top](https://srinimf.com/2018/05/27/adobe-flex-quiz-questions/) commercial Hadoop platforms [read here.](http://data-informed.com/10-top-commercial-hadoop-platforms/)

Related[My list Top 10 Sqoop Interview Questions asked recently](https://srinimf.com/2017/08/28/10-sqoop-interview-questions/)

### Share this:

-
- [Tweet](https://twitter.com/share)
- [![](https://i0.wp.com/assets.pinterest.com/images/pidgets/pinit_fg_en_rect_gray_20.png)](https://www.pinterest.com/pin/create/link/?url=https%3A%2F%2Fsrinimf.com%2F2017%2F08%2F22%2Fall-about-hadoop-eco-system-and-its-popular-technologies%2F&media=https%3A%2F%2Fi0.wp.com%2Fsrinimf.com%2Fwp-content%2Fuploads%2F2017%2F08%2Fpexels-photo-2228585.jpeg%3Ffit%3D1200%252C800%26ssl%3D1&description=Hadoop%20Technologies%20Explained%3A%20From%20MapReduce%20to%20HDFS)
-
- [Share on Tumblr](https://www.tumblr.com/share)
- [Share on WhatsApp (Opens in new window) WhatsApp](https://srinimf.com/2017/08/22/all-about-hadoop-eco-system-and-its-popular-technologies/?share=jetpack-whatsapp)
- [More](#)
-

- [Print (Opens in new window) Print](https://srinimf.com/2017/08/22/all-about-hadoop-eco-system-and-its-popular-technologies/?share=print)
- [Share on Reddit (Opens in new window) Reddit](https://srinimf.com/2017/08/22/all-about-hadoop-eco-system-and-its-popular-technologies/?share=reddit)
- [Share on Telegram (Opens in new window) Telegram](https://srinimf.com/2017/08/22/all-about-hadoop-eco-system-and-its-popular-technologies/?share=telegram)
- [Email a link to a friend (Opens in new window) Email](mailto:?subject=%5BShared%20Post%5D%20Hadoop%20Technologies%20Explained%3A%20From%20MapReduce%20to%20HDFS&body=https%3A%2F%2Fsrinimf.com%2F2017%2F08%2F22%2Fall-about-hadoop-eco-system-and-its-popular-technologies%2F&share=email)
-

### Like this:

Like Loading…