Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/vitalibo/hadoop-quickstart-archetype
https://github.com/vitalibo/hadoop-quickstart-archetype
hadoop maven-archetype
Last synced: about 1 month ago
JSON representation
- Host: GitHub
- URL: https://github.com/vitalibo/hadoop-quickstart-archetype
- Owner: vitalibo
- Created: 2017-02-26T21:43:14.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2017-02-28T00:26:11.000Z (almost 8 years ago)
- Last Synced: 2024-11-07T09:48:49.975Z (3 months ago)
- Topics: hadoop, maven-archetype
- Language: Java
- Size: 9.77 KB
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Quickstart Hadoop Project Archetype
## Overview
The project is a Maven archetype for Apache Hadoop. This archetype allows to quickly create Maven project to develop an Hadoop application.
### Structure
```
project
|-- .gitignore
|-- deploy.json
|-- pom.xml
`-- src
|-- main
| |-- java
| | |-- driver
| | | `-- WordCountDriver.java
| | |-- mapper
| | | `-- TokenizerMapper.java
| | `-- reducer
| | `-- IntSumReducer.java
| `-- resources
| `-- log4j.properties
`-- test
|-- java
| `-- WordCountMapReduceTest.java
`-- resources
```
### Dependencies
- JDK (v1.7)
- Apache Hadoop & YARN (v2.7.3)
- Log4j (v1.2.17)
- Lombok (v1.16.12)
- JUnit (v4.12)
- MRUnit (v1.1.0)
- [SSH Maven Plugin](https://github.com/vitalibo/ssh-maven-plugin) (v1.7.3)
## How to use
To install the archetype in your local repository please use following script
```
git clone [email protected]:vitalibo/hadoop-quickstart-archetype.git
cd hadoop-quickstart-archetype
mvn clean install
```
To generate new project based on this archetype use following command
```
mvn archetype:generate \
-DarchetypeGroupId=com.github.vitalibo \
-DarchetypeArtifactId=hadoop-quickstart-archetype \
-DarchetypeVersion=0.1.0-SNAPSHOT \
-DgroupId=your.company \
-DartifactId=project-name \
-Dssh-server-name=hdp2.4
```
more information about ssh-server-name please see [here](https://github.com/vitalibo/ssh-maven-plugin)