https://github.com/linkdd/emapred

Erlang Map-Reduce framework
https://github.com/linkdd/emapred

distributed erlang mapreduce

Last synced: 19 days ago
JSON representation

Erlang Map-Reduce framework

• Host: GitHub
• URL: https://github.com/linkdd/emapred
• Owner: linkdd
• License: apache-2.0
• Created: 2019-03-28T11:04:47.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2019-03-30T05:35:27.000Z (almost 6 years ago)
• Last Synced: 2024-11-05T10:38:08.304Z (2 months ago)
• Topics: distributed, erlang, mapreduce
• Language: Erlang
• Size: 18.6 KB
• Stars: 0
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE

Awesome Lists containing this project

README

        
emapred

=======

Erlang distributed Map-Reduce framework.

.. image:: https://travis-ci.org/linkdd/emapred.svg?branch=master

    :target: https://travis-ci.org/linkdd/emapred

Features

--------

 - run arbitrary Map/Reduce functions on a list

 - automatically distribute workload across nodes running this application

Example

-------

.. code-block:: erlang

   {ok, P} = emapred_pipeline:new(

       % Mapper

       fun(E) ->

           case E > 5 of

               true  -> {emit, {foo, 1}};

               false -> {emit, {foo, -1}}

           end

       end,

       % Reducer

       fun(_Key, Increment, Counter) ->

           {ok, Counter + Increment}

       end,

       % Initial value for reducer accumulator

       0

   ),

   % Stream elements to map

   ok = emapred_pipeline:send(P, 5),

   ok = emapred_pipeline:send(P, 6),

   ok = emapred_pipeline:send(P, 3),

   ok = emapred_pipeline:send(P, 4),

   ok = emapred_pipeline:send(P, 7),

   % Stop streaming and get reduced result

   -1 = emapred_pipeline:stop(P).

Build

-----

.. code-block:: console

   $ rebar3 compile