https://github.com/ihor/phadoop

Map/reduce jobs for Hadoop in PHP
https://github.com/ihor/phadoop

hadoop map-reduce php

Last synced: 2 months ago
JSON representation

Map/reduce jobs for Hadoop in PHP

• Host: GitHub
• URL: https://github.com/ihor/phadoop
• Owner: ihor
• License: mit
• Created: 2011-10-28T20:26:25.000Z (about 13 years ago)
• Default Branch: master
• Last Pushed: 2016-01-03T23:12:36.000Z (about 9 years ago)
• Last Synced: 2024-04-17T07:03:51.620Z (9 months ago)
• Topics: hadoop, map-reduce, php
• Language: PHP
• Homepage:
• Size: 77.1 KB
• Stars: 18
• Watchers: 3
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
Phadoop

=======

Phadoop allows you to write map/reduce tasks for Hadoop in PHP. I created it to give a techtalk about Hadoop in the company I worked in. It is not ready for production use yet but can help you to play with Hadoop in PHP.

Installation

------------

Define the following requirement in your composer.json file:

```

"require": {

    "ihor/phadoop": "0.1.x-dev"

}

```

or simply execute the following in the command line:

```

composer require ihor/phadoop

```

Usage

-----

```php

class Mapper extends \Phadoop\MapReduce\Job\Worker\Mapper

{

    protected function map($key, $value)

    {

        $this->emit('wordsNumber', count(preg_split('/\s+/', trim((string) $value))));

    }

}

class Reducer extends \Phadoop\MapReduce\Job\Worker\Reducer

{

    protected function reduce($key, \Traversable $values)

    {

        $result = 0;

        foreach ($values as $value) {

            $result += (int) $value;

        }

        $this->emit($key, $result);

    }

}

$mr = new \Phadoop\MapReduce('');

$job = $mr->createJob('WordCounter', 'Temp')

    ->setMapper(new Mapper())

    ->setReducer(new Reducer())

    ->clearData()

    ->addTask('Hello World')

    ->addTask('Hello Hadoop')

    ->putResultsTo('Temp/Results.txt')

    ->run();

echo $job->getLastResults();

```

You can find more examples in the [examples](https://github.com/ihor/Phadoop/tree/master/examples) directory.