{"id":19063236,"url":"https://github.com/delner/butterfli","last_synced_at":"2026-06-23T16:31:07.855Z","repository":{"id":34281845,"uuid":"38170253","full_name":"delner/butterfli","owner":"delner","description":"Provides a common container for social media objects.","archived":false,"fork":false,"pushed_at":"2016-05-21T18:36:25.000Z","size":125,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-11-30T14:44:06.714Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/delner.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-06-27T18:15:02.000Z","updated_at":"2016-05-21T18:36:29.000Z","dependencies_parsed_at":"2022-09-14T02:31:53.074Z","dependency_job_id":null,"html_url":"https://github.com/delner/butterfli","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/delner/butterfli","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/delner%2Fbutterfli","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/delner%2Fbutterfli/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/delner%2Fbutterfli/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/delner%2Fbutterfli/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/delner","download_url":"https://codeload.github.com/delner/butterfli/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/delner%2Fbutterfli/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34698687,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-23T02:00:07.161Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T00:29:21.209Z","updated_at":"2026-06-23T16:31:07.832Z","avatar_url":"https://github.com/delner.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Butterfli](http://cdn.delner.com/www/images/projects/butterfli/logo_small.svg)\n==========\n\n[![Build Status](https://travis-ci.org/delner/butterfli.svg?branch=master)](https://travis-ci.org/delner/butterfli) ![Gem Version](https://img.shields.io/gem/v/butterfli.svg?maxAge=2592000)\n###### *For Ruby 1.9.3, 2.0.0, 2.1.0*\n\n### Introduction\n\n`butterfli` is a gem for developers who want quick-access to data from popular APIs. It takes data from well-known endpoints (e.g. Instagram, Twitter, etc.) and converts them into a common container, known as a `Story`.\n\nCurrently supported providers include:\n - [Instagram](https://github.com/delner/butterfli-instagram)\n\nFuture support for providers include:\n - Facebook\n - Twitter\n - Google+\n - Vine\n - Foursquare\n\nIt is the base gem for the full *Butterfli* suite:\n\n**Core gems**:\n - [`butterfli`](https://github.com/delner/butterfli): Core gem for Butterfli suite.\n - [`butterfli-rails`](https://github.com/delner/butterfli-rails): Core gem for Rails-engine based API interactions.\n\n**Extension gems**:\n - [`butterfli-instagram`](https://github.com/delner/butterfli-instagram): Adds Instagram data to the Butterfli suite.\n - [`butterfli-instagram-rails`](https://github.com/delner/butterfli-instagram-rails): Adds Rails API endpoints for realtime-subscriptions.\n\n### Installation\n\nInstall the gem via `gem install butterfli`\n\n### Usage\n\nThe base Butterfli gem is for developers who want to extend the Butterfli suite. To use the gem to access some popular APIs, check out the *Extension gems* list above.\n\n### Features\n\nButterfli has several core components which support data retrieval and processing tasks. Some of these components can be configured directly to change Butterfli's behavior, and others are only for use by extension gems explicitly.\n\n#### Jobs\n\nMost data retrieval and processing can be performed in atomic units of work. Butterfli segments these operations into `Job`s, which have specific input and output. All jobs inherit from `Butterfli::Jobs::Job`, respond to `#work`, and return output.\n\n```ruby\njob = Butterfli::Jobs::Job.new\njob.work\n```\n\nThe most common job type is the `Butterfli::Jobs::StoryJob`, which returns a collection of `Butterfli::Data::Story`. It will also send the stories it retrieves to any configured *writers*: an endpoint like a database, cache, or event handler which accepts `Butterfli::Jobs::StoryJob`. (See the **Writing** section for more details.)\n\n#### Asynchronous processing\n\nJobs can also be processed using the Butterfli processor (disabled by default.) When enabled, this processor can enqueue a `Butterfli::Jobs::StoryJob` and call `#work` on them asynchronously.\n\n```ruby\nButterfli.processor.enqueue(job)\n```\n\nThis can be especially useful for expensive data retrieval or improving application responsiveness. The back-end of the processor can be configured to use a variety of different strategies for processing jobs.\n\n##### Monolith processor\n\nThe in-application processor provided by Butterfli is called the *monolith processor*. It is a simple thread pool which will run jobs within the application that queues them. This mode is useful for simple, lightweight deployments that do not require much processing power, or multiple nodes.\n\nTo setup the monolith processor, add the following to the Butterfli configuration:\n\n```ruby\nButterfli.configure do |config|\n  config.processor :monolith do |processor|\n    # Defines what a worker should do after completing a batch of jobs.\n    #  - sleep: puts the worker to sleep for a fixed period of time.\n    #  - block: sleeps the worker indefinitely until a job is queued, or its signaled to wake up.\n    #  (Default: :block)\n    processor.after_work = :sleep\n    # Number of threads to use. (Default: 1)\n    processor.num_workers = 1\n    # If after_work = :sleep, interval to sleep for, in seconds. (Default: 5)\n    processor.sleep_for = 5\n  end\nend\n```\n\nThen at the bottom of your configuration file, add the following:\n\n```ruby\n# Starts the monolith processor worker threads\nButterfli.processor.adapter.start\n```\n\nYou can also listen to specific work events for monitoring or logging purposes:\n\n```ruby\nButterfli.configure do |config|\n  config.processor :monolith do |processor|\n    # Invoked for individual jobs\n    processor.on_job_started do |job|\n    end\n    processor.on_job_completed do |job, result|\n    end\n    processor.on_job_error do |job, error|\n    end\n\n    # Invoked for a batch of jobs\n    processor.on_work_started do\n    end\n    processor.on_work_completed do |result|\n    end\n    processor.on_work_error do |error|\n    end\n    \n    # Invoked if the worker inexplicably dies\n    processor.on_worker_death do |error|\n    end\n  end\nend\n```\n\n#### Writing\n\nA place where output from Butterfli is sent is known as a *writer*: an endpoint to which data can be written. It could be a database, cache, file, function, API, etc...\n\nEach writer listens on *channels*. A channel expects a specific kind of input, and writes to its destination accordingly. The default channel is `:stories`, which expects a collection of `Butterfli::Data::Story`.\n\nAn example of writing stories:\n\n```ruby\n# Does not provide error handling, and might raise errors\nButterfli.writers.each { |w| w.write(:stories, stories) }\n# Provides error handling for any StandardError\nButterfli.writers.each { |w| w.write_with_error_handling(:stories, stories) }\n```\n\nThe `#write_with_error_handling` allows the developer to handle write errors via an event handler.\n\nTo handle these errors, add the following to your Butterfli configuration:\n\n```ruby\nButterfli.configure do |config|\n  config.writer :example do |writer|\n    writer.on_write_error do |error, stories|\n      # Custom error handling code here.\n    end\n  end\nend\n```\n\n##### Syndication writer\n\nYou can push events directly through Butterfli's event-handling system to other parts of application. (See **Syndication** for more details.)\n\nTo write stories to these subscribers, add the following to the Butterfli configuration:\n\n```ruby\nButterfli.configure do |config|\n  config.writer :syndicate\nend\n```\n\n#### Caching\n\nSome components in Butterfli require caching to operate (particularly for pagination and throttling features.) It provides a cache-layer from which simple values can be read and written.\n\nSome basic operations include:\n\n```ruby\nButterfli.cache.read(key)\nButterfli.cache.write(key, value)\nButterfli.cache.fetch(key, value)\nButterfli.cache.delete(key)\n```\n\nThe storage mechanism for this cache is configurable for your application's needs.\n\n##### Memory cache\n\nThe default cache is an in-memory `Hash` for all values. This works well for simple applications that do not require much memory, or multiple nodes.\n\nYou need not set any configuration to use this cache. However, you can explicitly enable it in the Butterfli configuration using:\n\n```ruby\nButterfli.configure do |config|\n  config.cache :memory\nend\n```\n\n#### Syndication\n\nButterfli provides a simple, in-application event-handling system called *syndication*. It allows any part of the application to listen for new stories.\n\nIn your application, you can listen for stories using `#subscribe`, to register an event handler:\n```ruby\nButterfli.subscribe do |stories|\n  puts \"I received #{stories.length} stories!\"\nend\n```\n\nThe above block will be called when any kind of story is received. To send stories to subscribers, you can invoke:\n\n```ruby\nButterfli.syndicate(stories)\n```\n\nYou can also subscribe to specific types of stories. Using `to:` allows you to specify where the story came from (e.g. `:instagram`) and `type:` allows you to specify what kind of story it should be.\n```ruby\nButterfli.subscribe to: :instagram, type: :image do |stories|\n  puts \"I received #{stories.length} image stories!\"\nend\n```\n\n#### Regulation\n\n*(TODO)*\n\n### Changelog\n\n#### Version 0.0.1\n\n - Initial version of Butterfli (defines Story common container)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdelner%2Fbutterfli","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdelner%2Fbutterfli","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdelner%2Fbutterfli/lists"}