https://github.com/bruceadams/pmap

Convenience Parallel Processing methods for Ruby
https://github.com/bruceadams/pmap

concurrency hacktoberfest multithreading parallel peach pmap ruby ruby-threads thread

Last synced: 5 days ago
JSON representation

Convenience Parallel Processing methods for Ruby

• Host: GitHub
• URL: https://github.com/bruceadams/pmap
• Owner: bruceadams
• License: apache-2.0
• Created: 2010-10-29T17:20:19.000Z (about 14 years ago)
• Default Branch: main
• Last Pushed: 2021-10-31T21:31:15.000Z (about 3 years ago)
• Last Synced: 2025-01-13T18:15:24.735Z (13 days ago)
• Topics: concurrency, hacktoberfest, multithreading, parallel, peach, pmap, ruby, ruby-threads, thread
• Language: Ruby
• Homepage:
• Size: 51.8 KB
• Stars: 70
• Watchers: 6
• Forks: 7
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

README

        
# pmap

[![Build status](https://github.com/bruceadams/pmap/actions/workflows/ruby.yml/badge.svg)](https://github.com/bruceadams/pmap/actions)

[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v1.4%20adopted-ff69b4.svg)](CODE_OF_CONDUCT.md)

[![Apache License](https://img.shields.io/github/license/bruceadams/pmap?logo=apache)](LICENSE)

This Ruby gem adds three methods to any Enumerable (notably including

any Array). The three added methods are:

* _pmap_ parallel map

* _peach_ parallel each

* _peach_with_index_ parallel each_with_index

Threading in Ruby has limitations

----------------------------------

Matz Ruby 1.8.* uses _green_ threads. All Ruby threads are run within

a single thread in a single process. A single Ruby program will never

use more than a single core of a multi-core machine.

Matz Ruby 1.9.* uses _native_ threads. Each Ruby thread maps directly

to a thread in the underlying operating system. In theory, a single

Ruby program can use multiple cores. Unfortunately, there is a global

interpreter lock _GIL_ that causes single-threaded behavior.

JRuby also uses _native_ threads. JRuby avoids the global interpreter

lock, allowing a single Ruby program to really use multiple CPU cores.

Threading useful for remote IO, such as HTTP

--------------------------------------------

Despite the Matz Ruby threading limitations, IO bound actions can

greatly benefit from multi-threading. A very typical use is making

multiple HTTP requests in parallel. Issuing those requests in separate

Ruby threads means the requests will be issued very quickly, well

before the responses start coming back. As responses come back, they

will be processed as they arrive.

Example

-------

Suppose that we have a function get_quote that calls out to a stock

quote service to get a current stock price. The response time for

get_quote ranges averages 0.5 seconds.

    stock_symbols = [:ibm, :goog, :appl, :msft, :hp, :orcl]

    # This will take about three seconds;

    # an eternity if you want to render a web page.

    stock_quotes = stock_symbols.map {|s| get_quote(s)}

    # Replacing "map" with "pmap" speeds it up.

    # This will take about half a second;

    # however long the single slowest response took.

    stock_quotes = stock_symbols.pmap {|s| get_quote(s)}

Thread Count

------------

The thread count defaults to 64 and is set based on `$pmap_default_thread_count`.

You can also set the thread count per call by passing it as an argument to the `pmap` and `peach` methods.

    # Use the default thread count (64)

    (1..128).peach { |i| sleep 1 } # Takes 2 seconds

    # Use a thread count of 128

    (1..128).peach(128) { |i| sleep 1 } # Takes 1 second

    # Use a thread count of 2

    (1..128).peach(2) { |i| sleep 1 } # Takes 64 seconds