https://github.com/jehiah/gomrjob

gomrjob - a Go Framework for Hadoop Map Reduce Jobs
https://github.com/jehiah/gomrjob

dataproc go hadoop mapreduce mrjob

Last synced: 10 days ago
JSON representation

gomrjob - a Go Framework for Hadoop Map Reduce Jobs

• Host: GitHub
• URL: https://github.com/jehiah/gomrjob
• Owner: jehiah
• License: mit
• Created: 2013-01-21T20:01:57.000Z (almost 12 years ago)
• Default Branch: master
• Last Pushed: 2024-08-05T19:49:52.000Z (3 months ago)
• Last Synced: 2024-10-14T13:59:04.584Z (23 days ago)
• Topics: dataproc, go, hadoop, mapreduce, mrjob
• Language: Go
• Homepage:
• Size: 583 KB
• Stars: 42
• Watchers: 7
• Forks: 4
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# GoMRJob 

![Build Status](https://github.com/jehiah/gomrjob/actions/workflows/ci.yaml/badge.svg?branch=master)

[![Docs](https://pkg.go.dev/badge/github.com/jehiah/gomrjob.svg)](http://pkg.go.dev/github.com/jehiah/gomrjob)

[![GitHub release](https://img.shields.io/github/release/jehiah/gomrjob.svg)](https://github.com/jehiah/gomrjob/releases/latest)

A Go framework for running Map Reduce Jobs on Hadoop.

http://godoc.org/github.com/jehiah/gomrjob

### Supported Configurations

* Hadoop with HDFS via `hadoop` CLI

* [Google Cloud Dataproc](https://cloud.google.com/dataproc/) with [Google Storage](https://cloud.google.com/storage/)

### About

This framework has been in production use at [Bitly](https://bitly.com/) since 2013, but it's light on examples. 

See the [example](./example) for more context.

Heavily inspired by [`Yelp/mrjob`](https://github.com/Yelp/mrjob)