https://github.com/paypal/nnanalytics

NameNodeAnalytics is a self-help utility for scouting and maintaining the namespace of an HDFS instance.
https://github.com/paypal/nnanalytics

fsimage hadoop hdfs metadata namespace scanner utility

Last synced: 4 days ago
JSON representation

NameNodeAnalytics is a self-help utility for scouting and maintaining the namespace of an HDFS instance.

• Host: GitHub
• URL: https://github.com/paypal/nnanalytics
• Owner: paypal
• License: apache-2.0
• Created: 2018-05-25T19:51:45.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2024-07-30T18:14:23.000Z (5 months ago)
• Last Synced: 2024-12-14T15:07:22.247Z (11 days ago)
• Topics: fsimage, hadoop, hdfs, metadata, namespace, scanner, utility
• Language: Java
• Homepage:
• Size: 2.64 MB
• Stars: 113
• Watchers: 24
• Forks: 71
• Open Issues: 17
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

README

        
[![License](http://img.shields.io/:license-Apache%202-blue.svg)](http://www.apache.org/licenses/LICENSE-2.0.txt)

[![Build Status](https://travis-ci.com/paypal/NNAnalytics.svg?branch=master)](https://travis-ci.com/paypal/NNAnalytics)

[![Documentation Status](https://readthedocs.org/projects/nnanalytics/badge/?version=latest)](https://nnanalytics.readthedocs.io/en/latest/)

[![codecov](https://codecov.io/gh/paypal/NNAnalytics/branch/master/graph/badge.svg)](https://codecov.io/gh/paypal/NNAnalytics)

[![Codacy Badge](https://api.codacy.com/project/badge/Grade/acc8afc858ff485ea67653b23c8ea82b)](https://github.com/paypal/NNAnalytics/pulls) 

[![Join the chat at https://gitter.im/NNAnalytics/Lobby](https://badges.gitter.im/NNAnalytics/Lobby.svg)](https://gitter.im/NNAnalytics/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

# 

"A Standby read-only HDFS NameNode, with no RPC server, that services clients over a REST API, utilizes Java 8 Stream API, all for the purpose of performing large and complicated scans of the entire file system metadata for end users."

Run a demo locally and instantly! Just run the following command on a workspace directory:

```

git clone https://github.com/paypal/NNAnalytics.git nna && cd ./nna && ./gradlew -PmainClass=org.apache.hadoop.hdfs.server.namenode.analytics.TestWithMiniClusterWithStreamEngine execute

```

Then go to http://localhost:4567 and you will have an NNA instance complete with a mini HA-enabled HDFS instance all updating in real time on your local machine!

__________________________________________________________________________________________________________________

# Architecture: Legacy vs NNA





__________________________________________________________________________________________________________________

# Documentation & Getting Started

  * [Click here to read the docs](http://nnanalytics.readthedocs.io/)

__________________________________________________________________________________________________________________

# Presentations

  * [SlideShare](https://www2.slideshare.net/PlamenJeliazkov/namenode-analytics-querying-hdfs-namespace-in-real-time)

  * [DataWorks Summit](https://www.youtube.com/watch?v=9xlB5C88tbk)

__________________________________________________________________________________________________________________