{"id":26241564,"url":"https://github.com/a6b8/feed-into-for-ruby","last_synced_at":"2026-04-21T18:33:30.352Z","repository":{"id":56846162,"uuid":"411519643","full_name":"a6b8/feed-into-for-ruby","owner":"a6b8","description":"Merge multiple different data streams into a custom structure. Also easy to expand by a custom module system.","archived":false,"fork":false,"pushed_at":"2023-04-13T01:37:38.000Z","size":14595,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-09T04:03:07.808Z","etag":null,"topics":["atom","curlai","data-mining","data-science","rss"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/a6b8.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-09-29T03:40:50.000Z","updated_at":"2022-01-01T18:41:57.000Z","dependencies_parsed_at":"2022-09-17T21:54:54.798Z","dependency_job_id":null,"html_url":"https://github.com/a6b8/feed-into-for-ruby","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a6b8%2Ffeed-into-for-ruby","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a6b8%2Ffeed-into-for-ruby/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a6b8%2Ffeed-into-for-ruby/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a6b8%2Ffeed-into-for-ruby/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/a6b8","download_url":"https://codeload.github.com/a6b8/feed-into-for-ruby/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243369726,"owners_count":20280061,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atom","curlai","data-mining","data-science","rss"],"created_at":"2025-03-13T08:32:11.831Z","updated_at":"2025-12-24T18:41:50.279Z","avatar_url":"https://github.com/a6b8.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com//a6b8/a6b8/main/assets/headlines/custom/feed-into-for-ruby.svg\" height=\"45px\" name=\"# Feed Into for Ruby\" alt=\"Feed Into for Ruby\"\u003e\n\u003c/a\u003e\n\nMerge multiple different data streams to a custom structure based on categories. Also easy to expand by a custom module system. \n\u003cbr\u003e\n\u003cbr\u003e\n\u003cbr\u003e\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/examples.svg\" height=\"45px\" name=\"examples\" alt=\"Examples\"\u003e\n\u003c/a\u003e\n\n**Merge multiple Streams**\n```ruby\nrequire 'feed_into'\n\nchannels_settings = {\n    name: :blockchain,\n    sym: :web,\n    options: {},\n    regexs: [ [ /https:\\/\\/your*website.com/ ] ],\n    download: :general,\n    mining: :rss_one,\n    pre: [],\n    transform: nil,\n    post: [ :pre_titles ]\n}\n \nfeeds = FeedInto::Group.new( \n    single: { channels: [ channels_settings ] } \n)\n\nurls = [\n    'https://your*website.com/1.xml',\n    'https://your*website.com/2.xml'\n]\n\nfeeds\n    .analyse( items: urls )\n    .merge\n    .to_rss( key: :unknown )\n```\n\u003cbr\u003e\n\n**Create .rss Categories from multiple Streams**\n```ruby\nrequire 'feed_into'\n\nchannels_settings = {\n    name: :blockchain,\n    sym: :web,\n    options: {},\n    regexs: [ [ /https:\\/\\/your*website.com/ ] ],\n    download: :general,\n    mining: :rss_one,\n    pre: [],\n    transform: nil,\n    post: [ :pre_titles ]\n}\n\nfeeds = FeedInto::Group.new( \n    single: { channels: [ channels_settings ] } \n)\n\nitem = [\n    {\n        name: 'Channel 1',\n        url: 'https://your*website.com/1.xml',\n        category: :nft\n    },\n    {\n        name: 'Channel 2',\n        url: 'https://your*website.com/2.xml',\n        category: :crypto\n    }\n]\n\nfeeds\n    .analyse( items: urls )\n    .merge\n    .to_rss_all\n```\n\u003cbr\u003e\n\u003cbr\u003e\n\u003ca href=\"#headline\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/table-of-contents.svg\" height=\"45px\" name=\"table-of-contents\" alt=\"Table of Contents\"\u003e\n\u003c/a\u003e\n\u003cbr\u003e\n\n1. [Examples](#examples)\u003cbr\u003e\n2. [Quickstart](#quickstart)\u003cbr\u003e\n3. [Setup](#setup)\u003cbr\u003e\n4. [Input Types](#input-types)\u003cbr\u003e\n   - [Single](#FeedIntosingle)\u003cbr\u003e\n     [String URL](#a-1-string-url)\u003cbr\u003e\n     [Hash Structure](#a2-hash-structure-cmd)\u003cbr\u003e\n   - Group\u003cbr\u003e\n     [Array of Strings](#b1-array-of-string)\u003cbr\u003e\n     [Array of Hashs](#b2-array-of-hash-cmds)\u003cbr\u003e\n5. [Methods](#methods)\u003cbr\u003e\n   - [Single](#FeedIntosingle-1)\u003cbr\u003e\n     [.analyse()](#analyse-item-)\u003cbr\u003e\n   - [Group](#FeedIntogroup)\u003cbr\u003e\n     [.analyse()](#analyse-items--silent-false-)\u003cbr\u003e\n     [.merge](#merge)\u003cbr\u003e\n     [.to_h()](#to_h-type-)\u003cbr\u003e\n     [.to_rss()](#to_rss-key-silent-)\u003cbr\u003e\n     [.to_rss_all](#to_rss_all-silent-)\u003cbr\u003e\n     [.status](#status)\u003cbr\u003e\n6. [Structure](#structure)\u003cbr\u003e\n7. [Options](#options)\u003cbr\u003e\n   - [Single](#FeedIntosingle-2)\u003cbr\u003e\n   - [Group](#FeedIntogroup-1)\u003cbr\u003e\n8. [Channels](#channels)\u003cbr\u003e\n   - [Settings Structure](#settings-structure)\n   - [Standard Components](#standard-components)\u003cbr\u003e\n   - [Custom Components](#custom-components)\u003cbr\u003e\n9. [Contributing](#contributing)\n10. [Limitations](#limitations)\n11. [Credits](#credits)\u003cbr\u003e\n12. [License](#license)\u003cbr\u003e\n13. [Code of Conduct](#code-of-conduct)\u003cbr\u003e\n14. [Support my Work](#support-my-work)\u003cbr\u003e\n\n\u003cbr\u003e\n\u003cbr\u003e\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/quickstart.svg\" height=\"45px\" name=\"quickstart\" alt=\"Quickstart\"\u003e\n\u003c/a\u003e\n\n```ruby\nrequire 'feed_into'\n\nchannels = [\n    {\n        name: :blockchain,\n        sym: :web,\n        options: {},\n        regexs: [ [ /https:\\/\\/your*website.com/ ] ],\n        download: :general,\n        mining: :rss_one,\n        pre: [],\n        transform: nil,\n        post: [ :pre_titles ]\n    }\n]\n\nfeed = FeedInto::Group.new( \n    single: { channels: channels } \n)\n\nurls = [ 'https://your*website.com/1.xml' ]\nfeed\n    .analyse( items: urls )\n    .status\n```\n\u003cbr\u003e\n\u003cbr\u003e\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/setup.svg\" height=\"45px\" name=\"setup\" alt=\"Setup\"\u003e\n\u003c/a\u003e\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'feed_into'\n```\n\nAnd then execute:\n\n    $ bundle install\n\nOr install it yourself as:\n\n    $ gem install feed_into\n\n\nOn Rubygems: \n- Gem: https://rubygems.org/gems/feed_into\n- Profile: https://rubygems.org/profiles/a6b8\n\n\u003cbr\u003e\n\u003cbr\u003e\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/input-types.svg\" height=\"45px\" name=\"input-types\" alt=\"Input Types\"\u003e\n\u003c/a\u003e\n\nA valid url string is required. If you use ```::Group``` you need to wrap your strings in an array. Consider to use a ```Hash Structure``` for best results.\n\n## FeedInto::Single\n\n2 types of inputs are allowed ```String``` and ```Hash```. \n- ```String``` must be a valid url.\n- ```Hash``` needs minimum an ```url:``` key with a valid url string. ```name:``` and ```category``` are optional.\n\n\u003cbr name=\"input-a-1\"\u003e\n\n### A. 1. ```String URL```\n\n**Input**\n```ruby\ncmd = 'https://your*website.com/1.xml'\nfeed.analyse( item: cmd )\n```\nUrl must be from type ```String``` and  a ```valid url```.\n\n**Internal Transformation to:**\n```ruby\n{\n    name: 'Unknown',\n    url: 'https://your*website.com/1.xml',\n    category: :unknown\n}\n```\n\n| **Name** | **Default** | **Description** |\n|------:|:------|:------|\n| **name:** | 'Unknown' | Set Name of Feed. If empty or not delivered the Name will set to 'Unknown' |\n| **category:** | :unknown | Set Category of Feed. If empty or not delivered the Category will set to :unknown |\n\nThe keys ```name:``` and ```category``` are required internally. If not set by the user both will be added with the default values: \"Unknown\" and :unknown. See [A.2.](#input-a-2) for more Informations\n\n\u003cbr name=\"input-a-2\"\u003e\n\n### A.2. ```Hash Structure``` (cmd)\n\n**Struct**\n```ruby\n{\n    name: String,\n    url: String,\n    category: Symbol\n}\n```\n\n**Example**\n```ruby\ncmd = {\n    name: 'Channel 1',\n    url: 'https://your*website.com/1.xml',\n    category: :nft\n}\n\nfeed.analyse( item: cmd )\n```\n\n**Validation**\n| **Name** | **Type / Regex** | **Required** | **Default** | **Description** |\n|------:|:------|:------|:------|:------|\n| **name:** | ```String``` | No | \"Unknown\" | Set Name of Feed. If empty or not delivered the Name will set to 'Channel 1' |\n| **url** | ```String``` and ```valid url``` | Yes |  | Set url of Feed. |\n| **category** | ```Symbol``` | No |  :unknown | Set Category of Feed. If empty or not delivered the Category will set to 'Channel 1' |\n\u003cbr\u003e\n\n## FeedInto::Group\n\n2 types of Arrays are allowed: ```Array of String``` or ```Array of Hash```.\n- ```Array of String``` must be a valid urls strings.\n- ```Array of Hash``` needs minimum an ```url:``` key with a valid url string per Hash.\n\n\u003cbr name=\"input-b-1\"\u003e\n\n### B.1. ```Array of String```\n\n**Example**\n```ruby\ncmds = [\n    'https://your*website.com/1.xml',\n    'https://your*website.com/2.xml'\n]\n\nfeeds.analyse( items: cmds )\n```\nValidation Info see [A.1.](#input-a-1)\n\n\u003cbr name=\"input-b-2\"\u003e\n\n### B.2. ```Array of Hash``` (cmds)\n\n**Example**\n```ruby\ncmds = [\n    {\n        name: 'Channel 1',\n        url: 'https://your*website.com/1.xml',\n        category: :nft\n    },\n    {\n        name: 'Channel 2',\n        url: 'https://your*website.com/2.xml',\n        category: :crypto\n    }\n]\n\nfeeds.analyse( items: cmds )\n```\nValidation Info see [A.2.](#input-a-2)\n\n\u003cbr\u003e\n\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/methods.svg\" height=\"45px\" name=\"methods\" alt=\"Methods\"\u003e\n\u003c/a\u003e\n\nThe methods are split in 2 classes \"Single\" and \"Group\". Single process only one url and inherit from Single and have all methods for bulk/group processing. For more details see [Structure](#structure).\n\n## FeedInto::Single\n### .new( modules: , options: )\nCreate a new Single Object to interact with.\n```ruby\nrequire 'feed_into'\n\nfeed = FeedInto::Single.new( \n    modules: './a/b/c/', \n    options: {}\n)\n```\n\n\n**Input**\n| **Name** | **Type** | **Required** | **Default** | **Example** | **Description** |\n|------:|:------|:------|:------|:------|:------| \n| **module folder** | ```String``` | No | ```nil``` | ```modules: './a/b/c/'``` | Set Module Folder path. |\n| **options** | ```Hash``` | No | ```{}``` | see [#options](#options) | Set options |\n\n\u003cbr\u003e\n\n### .analyse( item: )\nStart process of downloading, mining, modification and transforming based on your module setups.\n```ruby\nrequire 'feed_into'\n\nfeed = FeedInto::Single.new( \n    modules: './a/b/c/', \n    options: {}\n)\n\ncmd = {\n    name: 'Channel 1',\n    url: 'https://your*website.com/1.xml',\n    category: :crypto\n}\n\nfeed.analyse( item: cmd )\n\n# feed.analyse( item: 'https://your*website.com/1.xml' )\n```\n\n\n**Input**\n| **Name** | **Type** | **Required** | **Example** | **Description** |\n|------:|:------|:------|:------|:------| \n| **item** | ```String``` or ```Hash Structure``` (see [Input A.2.](#input-a-2)) | Yes | item: 'https://your*website.com/1.xml' | Insert Url by String or Hash Structure |\n\u003cbr\u003e\n\n## FeedInto::Group\n### .new( modules:, group:, single: )\nCreate a new Group Object to interact with.\n```ruby\nrequire 'feed_into'\n\nfeed = FeedInto::Group.new( \n    modules: './a/b/c/', \n    group: {},\n    single: {}\n)\n```\n\n\n**Input**\n| **Name** | **Type** | **Required** | **Default** | **Example** | **Description** |\n|------:|:------|:------|:------|:------|:------| \n| **module folder** | ```String``` | No | ```nil``` | ```modules: './a/b/c/'``` | Set Module Folder path. |\n| **group** | ```Hash``` | No | ```{}``` | see [Options](#options) | Set group options |\n| **single** | ```Hash``` | No | ```{}``` | see [Options](#options) | Set group options |\n\n**Return**\u003cbr\u003e\nHash    \n\u003cbr\u003e\n\n### .analyse( items: [], silent: false )\nStart process of bulk execution.\n```ruby\nrequire 'feed_into'\n\nfeed = FeedInto::Group.new( \n    modules: './a/b/c/', \n    group: {},\n    single: {}\n)\n\ncmds = [\n    {\n        name: 'Channel 1',\n        url: 'https://your*website.com/1.xml',\n        category: :nft\n    },\n    {\n        name: 'Channel 2',\n        url: 'https://your*website.com/2.xml',\n        category: :crypto\n    }\n]\n\nfeed.analyse( items: cmds )\n```\n\n\n**Input**\n| **Name** | **Type** | **Required** | **Default** | **Example** | **Description** |\n|------:|:------|:------|:------|:------|:------| \n| **items** | ```Array of String``` or ```Array of Hash``` | Yes | | See [Input B.1.](#input-b-1) and [B.2.](#input-b-1) for examples and more details. | Set Inputs URLs |\n| **silent** | ```boolean``` | No | ```false``` | silent: false | Print status messages |\n\n**Return**\u003cbr\u003e\nSelf\n\n\u003e To return result use ```.to_h```\n\n\u003cbr\u003e\n\n### .merge\nRe-arrange items by category and simplify data for rss output.\n\n```ruby\nrequire 'feed_into'\n\nfeed = FeedInto::Group.new( \n    modules: './a/b/c/', \n    group: {},\n    single: {}\n)\n\ncmds = [\n    {\n        name: 'Channel 1',\n        url: 'https://your*website.com/1.xml',\n        category: :nft\n    },\n    {\n        name: 'Channel 2',\n        url: 'https://your*website.com/2.xml',\n        category: :crypto\n    }\n]\n\nfeed\n    .analyse( items: cmds )\n    .merge\n```\n\n**Return**\u003cbr\u003e\nSelf\n\n\u003e To return result use ```.to_h```\n\n\u003cbr\u003e\n\n### .to_h( type: )\nOutput data to string.\n```ruby\nrequire 'feed_into'\n\nfeed = FeedInto::Group.new( \n    modules: './a/b/c/', \n    group: {},\n    single: {}\n)\n\ncmds = [\n    {\n        name: 'Channel 1',\n        url: 'https://your*website.com/1.xml',\n        category: :nft\n    },\n    {\n        name: 'Channel 2',\n        url: 'https://your*website.com/2.xml',\n        category: :crypto\n    }\n]\n\nfeed\n    .analyse( items: cmds )\n    .merge\n    .to_h( type: :analyse ) \n```\n\n\n**Input**\n| **Name** | **Type** | **Required** | **Default** | **Example** | **Description** |\n|------:|:------|:------|:------|:------|:------| \n| **type** | ```Symbol``` | No | ```nil``` | ```:analyse``` or ```:merge``` | Define explizit which hash should be returned. If not set .to_h will return ```:merge``` if not nil otherwise ```:analyse``` |\n\n**Return**\u003cbr\u003e\nHash    \n\u003cbr\u003e\n\n### .to_rss( key:, silent: )\nOutput a ```.merge()``` category to a valid rss feed.\n\n```ruby\nrequire 'feed_into'\n\nfeed = FeedInto::Group.new( \n    modules: './a/b/c/', \n    group: {},\n    single: {}\n)\n\ncmds = [\n    {\n        name: 'Channel 1',\n        url: 'https://your*website.com/1.xml',\n        category: :nft\n    },\n    {\n        name: 'Channel 2',\n        url: 'https://your*website.com/2.xml',\n        category: :crypto\n    }\n]\n\nfeed\n    .analyse( items: cmds )\n    .merge\n    .to_rss( key: :analyse ) \n```\n\n\n**Input**\n| **Name** | **Type** | **Required** | **Default** | **Example** | **Description** |\n|------:|:------|:------|:------|:------|:------| \n| **key** | ```Symbol``` | Yes | ```nil``` | :nft | Only a single category will be transformed to rss. Define category here. |\n| **silent** | ```Boolean``` | No | ```false``` | | Print status messages |\n\n**Return**\u003cbr\u003e\nHash    \n\u003cbr\u003e\n\n### .to_rss_all( silent: )\nOutput ```.merge()``` categories to a valid rss feeds.\n```ruby\nrequire 'feed_into'\n\nfeed = FeedInto::Group.new( \n    modules: './a/b/c/', \n    group: {},\n    single: {}\n)\n\ncmds = [\n    {\n        name: 'Channel 1',\n        url: 'https://your*website.com/1.xml',\n        category: :nft\n    },\n    {\n        name: 'Channel 2',\n        url: 'https://your*website.com/2.xml',\n        category: :crypto\n    }\n]\n\nfeed\n    .analyse( items: cmds )\n    .merge\n    .to_rss_all \n```\n\n\n**Input**\n| **Name** | **Type** | **Required** | **Default** | **Example** | **Description** |\n|------:|:------|:------|:------|:------|:------| \n| **silent** | ```Boolean``` | No | ```false``` | | Print status messages |\n\n**Return**\u003cbr\u003e\nHash    \n\u003cbr\u003e\n\n\n### .status\nOutputs useful informations about the ```.analyse()``` pipeline.\n```ruby\nrequire 'feed_into'\n\nfeed = FeedInto::Group.new( \n    modules: './a/b/c/', \n    group: {},\n    single: {}\n)\n\ncmds = [\n    {\n        name: 'Channel 1',\n        url: 'https://your*website.com/1.xml',\n        category: :nft\n    },\n    {\n        name: 'Channel 2',\n        url: 'https://your*website.com/2.xml',\n        category: :crypto\n    }\n]\n\nfeed\n    .analyse( items: cmds )\n    .status\n```\n\n\n**Input**\n| **Name** | **Type** | **Required** | **Default** | **Example** | **Description** |\n|------:|:------|:------|:------|:------|:------| \n| **silent** | ```Boolean``` | No | ```false``` | | Print status messages |\n\n**Return**\u003cbr\u003e\nHash    \n\n\u003cbr\u003e\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/structure.svg\" height=\"45px\" name=\"structure\" alt=\"Structure\"\u003e\n\u003c/a\u003e\n\nClass Overview\n\n```\nFeedInto::Single\nFeedInto::Group\n\n--\u003e CLASS: Group\n    ---------------------------------------\n    |  - new( modules:, sgl:{}, grp:{} )  |\n    |  - analyse( items:, silent: false ) |\n    |  - merge                            |\n    |  - to_h( type: nil )                |\n    |  - to_rss( key: Symbol )            |\n    |  - to_rss_all( silent: false )      |\n    |                                     |\n------\u003e CLASS: Single                     |\n    |   --------------------------------  |\n    |   |  - new( modules:, opts:{} ) \u003c---- MODULE FOLDER\n    |   |  - analyse( item: )          |  |\n    |   |                              |  |\n    |   |   FUNCTIONS: General         |  |\n    |   |   -------------------------  |  |\n    |   |   |  - crl_general        |  |  |\n    |   |   |   :download           |  |  |\n    |   |   |   :pre_titles         |  |  |\n    |   |   |   :mining_rss_one     |  |  |\n    |   |   |   :mining_rss_two     |  |  |\n    |   |   |   :format_url_s3      |  |  |\n    |   |   |   :format_html_remove |  |  |\n    |   |   -------------------------  |  |\n    |   --------------------------------  |  \n    ---------------------------------------\n```\n\nCustom Modules\n```\n\n    MODULE FOLDER \"./a/b/c/\"\n    -----------------------------------------------\n    |                                             |\n    |   MODULE: #{Module_Name}                    |\n    |    FILE:  #{module_name}.rb                 |\n    |   -------------------------------------     |\n    |   |  Required:                        |     |\n    |   |  - crl_#{module_name}             |---  |\n    |   |  - crl_#{module_name}_settings    |  |  | \n    |   |                                   |  |  | \n    |   |  Custom:                          |  |  |\n    |   |  - crl_#{module_name}_custom_name |  |  |\n    |   -------------------------------------  |  |\n    |      |                                   |  |\n    |      -------------------------------------  |\n    |                                             |\n    -----------------------------------------------\n```\nSee [Channels](#channels) for more details.\n\n\u003c/a\u003e\n\u003cbr\u003e\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/options.svg\" height=\"45px\" name=\"options\" alt=\"Options\"\u003e\n\u003c/a\u003e\n\nOptions are split in 2 section: Single and Group. \n\n- In ```::Single``` use ```.new( ... options: )``` to set options.\n- In ```::Group``` use ```.new( ... single:, group: )``` to set options.\n\n**Example**\n```ruby\noptions = {\n    single: {\n        format__title__symbol__vide: \"🐨\",\n        format__title__symbol__custom: \"👽\"\n    },\n    group: {\n        sleep__scores__user__value: 5,\n        sleep__scores__server__value: 10\n    }\n}\n\n# Single\nfeed = FeedInto::Single.new( \n    modules: './a/b/c/',\n    options: options[:single]\n)\n\n# Group\nfeeds = FeedInto::Group.new( \n    modules: './a/b/c/',\n    single: options[:single],\n    group: options[:group]\n)\n```\n\n## FeedInto::Single\n\n| Nr | Name | Key | Default | Type | Description |\n| :-- | :-- | :-- | :-- | :-- | :-- |\n| 1. | Title Symbol Video |:format__title__symbol__video | `\"👾\"` | String | Set Symbol for Video, used in :pre_title |\n| 2. | Title Symbol Custom |:format__title__symbol__custom | `\"⚙️ \"` | String | Set Symbol for Custom, used in :pre_title |\n| 3. | Title Symbol Web |:format__title__symbol__web | `\"🤖\"` | String | Set Symbol for Web, used in :pre_title |\n| 4. | Title Separator |:format__title__separator | `\"\\|\"` | String | Change separator, used in :pre_title |\n| 5. | Title More |:format__title__more | `\"...\"` | String | Used in :pre_title |\n| 6. | Title Length |:format__title__length | `100` | Integer | Set a maximum length, used in :pre_title |\n| 7. | Title Str |:format__title__str | `\"{{sym}} {{cmd_name__upcase}} ({{channel_name__upcase}}) {{separator}} {{title_item__titleize}}\"` | String | Set Title Structure, used in :pre_title |\n| 8. | Download Agent |:format__download__agent | `\"\"` | String | Set a Agent for Header Request. Use {version} to generate a random version. |\n\n## FeedInto::Group\n\n| Nr | Name | Key | Default | Type | Description |\n| :-- | :-- | :-- | :-- | :-- | :-- |\n| 1. | Range | :sleep__range | `15` | Integer | Set how many items are relevant to calculate score for sleeping time. |\n| 2. | Varieties |:sleep__varieties | `[{:variety=\u003e1, :sleep=\u003e2}, {:variety=\u003e2, :sleep=\u003e1}, {:variety=\u003e3, :sleep=\u003e0.5}, {:variety=\u003e4, :sleep=\u003e0.25}, {:variety=\u003e5, :sleep=\u003e0.15}, {:variety=\u003e6, :sleep=\u003e0.1}]` | Array | Set diffrent sleep times by diffrent variety levels |\n| 3. | Scores Ok Value |:sleep__scores__ok__value | `0` | Integer | Sleeping Time for :ok download. |\n| 4. | Scores User Value |:sleep__scores__user__value | `1` | Integer | Sleeping Time for :user download errors. |\n| 5. | Scores Server Value |:sleep__scores__server__value | `3` | Integer | Sleeping Time for :server download errors. |\n| 6. | Scores Other Value |:sleep__scores__other__value | `0` | Integer | Sleeping Time for :other download errors. |\n| 7. | Stages |:sleep__stages | `[{:name=\u003e\"Default\", :range=\u003e[0, 2], :skip=\u003efalse, :sleep=\u003e0}, {:name=\u003e\"Low\", :range=\u003e[3, 5], :skip=\u003efalse, :sleep=\u003e2}, {:name=\u003e\"High\", :range=\u003e[6, 8], :skip=\u003efalse, :sleep=\u003e5}, {:name=\u003e\"Stop\", :range=\u003e[9, 999], :skip=\u003etrue}]` | Array | Set Sleep range for diffrent scores. |\n\n\u003cbr\u003e\n\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/channels.svg\" height=\"45px\" name=\"channels\" alt=\"Channels\"\u003e\n\u003c/a\u003e\n\nTo recognize an url, a \"channel\" must be created. A channel requires a ```Hash``` which defines the pipeline for the given regex urls. You don´t need to write your own module if you use the standard components. To extend the functionalities you can write your own module and initialize by refer to your module folder.\n\n\n## Settings Structure\n\nEvery Channel need a Settings Structure to get recognized.\n```ruby\n{\n    name: Symbol,\n    sym: Symbol,\n    options: Hash,\n    regexs: Nested Array,\n    download: Symbol,\n    mining: Symbol,\n    pre: Array of Symbols,\n    transform: Symbol,\n    post: Array of Symbols\n}\n```\n\n\n| **Name** | **Type** | **Required** | **Example** | **Description** |\n|------:|:------|:------|:------|:------|\n| **name** | ```Symbol``` | Yes | ```:module_name``` | Set your unique channel name as symbol class |\n| **sym** | ```Symbol``` | Yes |  ```:web``` | Assign a category sym to your channel. See [Options](#options) for more details. |\n| **options** | ```Hash``` | Yes |  ```{ length: 23 }``` | Set specific channel variable here |\n| **regexs** | ```Nested Array``` | Yes | ```[ [ /https:\\/\\/module_name/ ] ]```| To assign a given url to your channel use an Array (with multiple regexs) and wrap them in an Array. All Regexs from only **one** array must be true. |\n| **download** | ```Symbol``` | Yes | ```:general``` | Select which 'download' method you prefer. |\n| **mining** | ```Symbol``` | Yes | ```:rss_one``` | Select which 'mining' method you prefer. |\n| **pre** | ```Array``` | Yes |  ```[]``` | Select which 'pre' methods you prefer. |\n| **transform** | ```Symbol``` | ```nil``` |  | Select which 'transform' methods you prefer. |\n| **post** | ```Array``` | Yes |  ```[ :pre_titles ]``` | Select which 'post' methods you prefer. |\n\n\n## Standard Components\nInject a struct with **only** standard components in this way. You can find more informations about the available components in [Structure](#structure)\n\n```ruby\nrequire 'feed_into'\n\nchannels_settings = {\n    name: :blockchain,\n    sym: :web,\n    options: {},\n    regexs: [ [ /https:\\/\\/your*website.com/ ] ],\n    download: :general,\n    mining: :rss_one,\n    pre: [],\n    transform: nil,\n    post: [ :pre_titles ]\n}\n\nfeeds = FeedInto::Group.new( \n    single: { channels: [ channels_settings ] } \n)\n\nfeeds.analyse( items: [ 'https://your*website.com/1.xml' ] )\n\n# feed = FeedInto::Single.new( \n#     options: { channels: struct } \n# )\n# feed.analyse( item: 'https://your*website.com/1.xml' )\n```\n\n## Custom Components\n\nFor custom functionalities you need to define a Module. Use the following boilerplate for a quickstart. Please note:\n- Every function name starts with the prefix 'crl_'\n- The channel will be automatically initialized by search for 'crl_module_name_settings'.\n- Every pipeline contains five stages ```download```, ```mining```, ```pre```, ```transform```, ```post```. \n- The interaction with your Module is only over the function ```crl_module_name```. Delegate the traffic by a case statement.\n- For later tasks you should give back a least ```:title```, ```:url``` and ```[:time][:stamp]```. \n\n**Step 1:** Create Module\n\n./path/module_name.rb\n```ruby\nmodule ModuleName\n  def crl_module_name( sym, cmd, channel, response, data, obj )\n    messages = []\n\n    case sym\n      when :settings\n        data = crl_module_name_settings()\n      when :transform\n        data = crl_module_name_transform( data, obj, cmd, channel )\n    else\n      messages.push( \"module_name: #{sym} not found.\" )\n    end\n    \n    return data, messages\n  end\n  \n\n  private\n\n\n  def crl_module_name_settings()\n    {\n      name: :module_name,\n      sym: :video,\n      options: {},\n      regexs: [ [ /www.module_name.com/, /www.module_name.com/ ] ],\n      download: :general,\n      mining: :rss_two,\n      pre: [],\n      transform: :self,\n      post: [ :pre_titles ]\n    }\n  end\n\n  \n  def crl_module_name_transform( data, obj, cmd, channel )\n    data[:items] = data[:items].map do | item |\n        item = {\n            title: '',\n            time: { stamp: 1632702548 },\n            url: 'https://....'\n        }\n    end\n    return data\n  end\nend\n```\n\n\n**Step 2:** Initialize Module\n```ruby\nrequire 'feed_into'\n\nfeeds = FeedInto::Group.new( \n    modules: './path/'\n)\n\nfeeds\n    .analyse( items: [ 'module_name.com/rss' ] )\n    .merge\n    .rss_to_all\n```\n\u003cbr\u003e\n\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/contributing.svg\" height=\"45px\" name=\"contributing\" alt=\"Contributing\"\u003e\n\u003c/a\u003e\n\nBug reports and pull requests are welcome on GitHub at https:https://raw.githubusercontent.com/feed-into-for-ruby. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https:https://raw.githubusercontent.com/feed-into-for-ruby/blob/master/CODE_OF_CONDUCT.md).\n\n\u003cbr\u003e\n\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/limitations.svg\" height=\"45px\" name=\"limitations\" alt=\"Limitations\"\u003e\n\u003c/a\u003e\n\n- Proof of Concept, not battle-tested.\n\u003cbr\u003e\n\u003cbr\u003e\n\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/credits.svg\" height=\"45px\" name=\"credits\" alt=\"Credits\"\u003e\n\u003c/a\u003e\n\nThis gem use following gems:\n\n- [nokogiri](https://nokogiri.org)\n- [net/http](https://github.com/ruby/net-http)\n- [time](https://ruby-doc.org/core-2.6.3/Time.html)\n- [tzinfo](https://github.com/tzinfo/tzinfo)\n- [cgi](https://ruby-doc.org/stdlib-2.5.3/libdoc/cgi/rdoc/CGI.html)\n- json\n- [rss](https://github.com/ruby/rsshttps://github.com/ruby/rss)\n\n\u003cbr\u003e\n\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/license.svg\" height=\"45px\" name=\"license\" alt=\"License\"\u003e\n\u003c/a\u003e\n\nThe gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).\n\u003cbr\u003e\n\u003cbr\u003e\n\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/code-of-conduct.svg\" height=\"45px\" name=\"code-of-conduct\" alt=\"Code of Conduct\"\u003e\n\u003c/a\u003e\n    \nEveryone interacting in the feed-into-for-ruby project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https:https://raw.githubusercontent.com/feed-into-for-ruby/blob/master/CODE_OF_CONDUCT.md).\n\u003cbr\u003e\n\u003cbr\u003e\n\n\u003ca href=\"#table-of-contents\"\u003e\n\u003cimg href=\"#table-of-contents\" src=\"https://raw.githubusercontent.com/a6b8/a6b8/main/assets/headlines/default/star-us.svg\" height=\"45px\" name=\"star-us\" alt=\"Star Us\"\u003e\n\u003c/a\u003e\n\nPlease ⭐️ star this Project, every ⭐️ star makes us very happy!","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fa6b8%2Ffeed-into-for-ruby","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fa6b8%2Ffeed-into-for-ruby","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fa6b8%2Ffeed-into-for-ruby/lists"}