{"id":23085073,"url":"https://github.com/robotstudio/bors","last_synced_at":"2026-03-08T15:35:40.786Z","repository":{"id":32439631,"uuid":"133428260","full_name":"RobotStudio/bors","owner":"RobotStudio","description":"A highly flexible and versatile service integration framework.","archived":false,"fork":false,"pushed_at":"2024-12-07T07:08:21.000Z","size":284,"stargazers_count":2,"open_issues_count":16,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-08-16T04:39:12.429Z","etag":null,"topics":["data-ingestion","data-integration","scraper","scraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RobotStudio.png","metadata":{"files":{"readme":"README.rst","changelog":"HISTORY.rst","contributing":"CONTRIBUTING.rst","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.rst","dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-05-14T22:26:03.000Z","updated_at":"2023-11-21T06:20:40.000Z","dependencies_parsed_at":"2023-10-03T10:40:54.563Z","dependency_job_id":"eba2a68a-ecec-45b1-8dad-5ccd85f46319","html_url":"https://github.com/RobotStudio/bors","commit_stats":{"total_commits":215,"total_committers":4,"mean_commits":53.75,"dds":"0.39534883720930236","last_synced_commit":"8dc35f5d52c98e7e7140be8e3e9fb43ca6a4a70a"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/RobotStudio/bors","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobotStudio%2Fbors","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobotStudio%2Fbors/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobotStudio%2Fbors/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobotStudio%2Fbors/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RobotStudio","download_url":"https://codeload.github.com/RobotStudio/bors/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobotStudio%2Fbors/sbom","scorecard":{"id":121941,"data":{"date":"2025-08-11","repo":{"name":"github.com/RobotStudio/bors","commit":"8dc35f5d52c98e7e7140be8e3e9fb43ca6a4a70a"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3,"checks":[{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":10,"reason":"all changesets reviewed","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: GNU General Public License v3.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":0,"reason":"10 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-9hjg-9r4m-mvj7","Warn: Project is vulnerable to: GHSA-9wx4-h78v-vm56","Warn: Project is vulnerable to: PYSEC-2023-74 / GHSA-j8r2-6x86-q33q","Warn: Project is vulnerable to: GHSA-34jh-p97f-mpxf","Warn: Project is vulnerable to: PYSEC-2023-212 / GHSA-g4mx-q9vg-27p4","Warn: Project is vulnerable to: PYSEC-2020-149 / GHSA-hmv2-79q8-fv6g","Warn: Project is vulnerable to: GHSA-pq67-6m6q-mj2v","Warn: Project is vulnerable to: PYSEC-2021-108 / GHSA-q2q7-5pp4-w6pg","Warn: Project is vulnerable to: PYSEC-2023-192 / GHSA-v845-jxx5-vc9f","Warn: Project is vulnerable to: PYSEC-2020-148 / GHSA-wqvq-5m8c-6g24"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-16T02:38:58.785Z","repository_id":32439631,"created_at":"2025-08-16T02:38:58.785Z","updated_at":"2025-08-16T02:38:58.785Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274997333,"owners_count":25387930,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-13T02:00:10.085Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-ingestion","data-integration","scraper","scraping"],"created_at":"2024-12-16T17:48:53.873Z","updated_at":"2026-03-08T15:35:40.754Z","avatar_url":"https://github.com/RobotStudio.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":".. image:: https://travis-ci.org/RobotStudio/bors.svg?branch=master\n    :target: https://travis-ci.org/RobotStudio/bors\n\n.. image:: https://readthedocs.org/projects/bors/badge/?version=latest\n    :target: https://bors.readthedocs.io/en/latest/?badge=latest\n    :alt: Documentation Status\n\n.. image:: https://pyup.io/repos/github/RobotStudio/bors/shield.svg\n    :target: https://pyup.io/repos/github/RobotStudio/bors/\n    :alt: Updates\n\n.. image:: https://pyup.io/repos/github/RobotStudio/bors/python-3-shield.svg\n    :target: https://pyup.io/repos/github/RobotStudio/bors/\n    :alt: Python 3\n\n\nBors\n====\n\nA highly flexible and extensible service integration framework for\nscraping the web or consuming APIs.\n\nUsage\n=====\n\n1. Create your model based on the data you expect to incorporate.\n2. Decide on what you want to do with your data, and add it.\n3. Create or use an existing API integration library.\n4. Create your root application to tie it all together.\n\nObject Model\n------------\n\nWe use `marshmallow \u003chttps://marshmallow.readthedocs.io/en/latest/\u003e`__\nfor the underlying object schema definitions. Here's an example model:\n\n.. code:: python\n\n    from marshmallow import Schema, fields\n\n    class NewsItemSchema(Schema):\n        \"\"\"News item\"\"\"\n        id = f.Str(required=True)\n        url = f.Str(required=True)\n        title = f.Str(required=True)\n        pubDate = f.Str(required=True)\n        timestamp = f.Str(required=True)\n        feed_id = f.Int(required=True)\n        published_date = f.Str(required=True)\n        feed_name = f.Str(required=True)\n        feed_url = f.Str(required=True)\n        feed_enabled = f.Int(required=True)\n        feed_description = f.Str(required=True)\n        url_field = f.Str(required=True)\n        title_field = f.Str(required=True)\n        date_field = f.Str(required=True)\n        feed_image = f.Str(required=True)\n\nSee the ``marshmallow`` docs for more information.\n\nMiddleware Strategies\n---------------------\n\nMiddleware API is implemented in the form of strategies and follows this\nbasic layout:\n\n.. code:: python\n\n    \"\"\"\n    Simple context display strategy\n    \"\"\"\n\n    from bors.app.strategy import IStrategy\n\n\n    class Print(IStrategy):\n        \"\"\"Print Strategy implementation\"\"\"\n        def bind(self, context):\n            \"\"\"\n            Bind the strategy to the middleware pipeline,\n            returning the context\n            \"\"\"\n            print(f\"\"\"PrintStrategy: {context}\"\"\")\n\n            # just a pass-through\n            return context\n\nThe important things to note here: \\* We're inheriting from\n``IStrategy``. \\* We're implementing a ``bind`` method. \\* The bind\nmethod receives, potentially augments, and then returns the ``context``.\n\nAPI Integration\n---------------\n\nRequest Schema\n~~~~~~~~~~~~~~\n\nBecause our API is simple, we're going to use this as-is.\n\n.. code:: python\n\n    from bors.generics.request import RequestSchema\n\nResponse Schema\n~~~~~~~~~~~~~~~\n\nOur API sends us data in the following format:\n\n.. code:: json\n\n    {\n        \"data\": ...,\n        \"status\": \"OK\"\n    }\n\nFor this, we'll need to supplement a bit, removing the root fields and\nreturning the ``data`` value:\n\n.. code:: python\n\n    from marshmallow import fields\n    from bors.generics.request import ResponseSchema\n\n\n    class MyAPIResponseSchema(ResponseSchema):\n        \"\"\"Schema defining how the API will respond\"\"\"\n        status = fields.Str()\n        def get_result(self, data):\n            \"\"\"Return the actual result data\"\"\"\n            return data.get(\"data\", \"\")\n            \n        class Meta:\n            \"\"\"Add 'data' field\"\"\"\n            strict = True\n            additional = (\"data\",)\n\nAPI Class\n~~~~~~~~~\n\n.. code:: python\n\n    from bors.api.requestor import Req\n\n\n    class MyAPI(LoggerMixin):\n        name = \"my_api\"\n        def __init__(self, context):\n            self.create_logger()\n            \n            self.request_schema = RequestSchema\n            self.result_schema = MyAPIResponseSchema\n            self.context = context\n            \n            self.req = Req(\"http://some.api.endpoint/v1\", payload, self.log)\n            \n            # We don't need to deal directly with requests, so we pass them through\n            self.call = self.req.call\n        \n        def shutdown(self):\n            \"\"\"Perform last-minute stuff\"\"\"\n            pass\n\nHere we use the built-in ``Req`` class to issue requests to the API, we\nassign the ``request_schema`` and ``result_schema`` to classes in our\nobject, and we set the ``name``, ``context``, and ``call`` attributes.\nThe results passed through on the API are referencable from within the\nmiddleware context under the key ``my_api``.\n\nPulling it all together\n~~~~~~~~~~~~~~~~~~~~~~~\n\n.. code:: python\n\n    from bors.app.builder import AppBuilder\n    from bors.app.strategy import Strategy\n\n\n    def main():\n        strat = Strategy(Print())\n        app = AppBuilder([MyAPI], strat)\n        app.run()\n        \n    if __name__ == \"__main__\":\n        main()\n\nHere, we set as many strategies and API's as we want, then create and\nrun the ``app``.\n\nArchitecture\n============\n\n::\n\n      +------------+\n    +-+ MIDDLEWARE +------\u003e out\n    | +------------+\n    |                       API/WEB\n    | +------------+\n    +-+ PREPROCESS +\u003c------ in\n      +------------+\n\nAt its most basic level, a ``bors`` integrator engages with an\nintegration library (API) passing incoming data through a prepocessor to\ngenerate and validate incoming objects, then passes that data through\nmiddlewares. Outgoing interactions are initiated from within a\nmiddleware and passed directly to an API, allowing easily for\nrequest/response type behavior in addition to observe and react.\n\nIngesting Data\n--------------\n\n::\n\n          ^\n          |\n    +-----+------+\n    | MIDDLEWARE |\n    +-----+------+\n          ^\n    +-----+------+\n    | PREPROCESS |\n    +-----+------+\n          ^\n          |\n          +\n         API/\n         WEB\n\nIngested data provokes calls along the pipeline.\n\nOutgoing Data\n-------------\n\n::\n\n         API/\n         WEB\n          ^\n          |\n    +-----+------+\n    | MIDDLEWARE |\n    +------------+\n\nEnacted events stimulate API or web actions.\n\nPreprocessing\n-------------\n\nPreprocessing is nothing more than an object-ization of the incoming\ndata. This provides two benefits: 1. Data can be generalized across API\ninterfaces. 2. Data structure can be validated and enforced.\n\nMiddlewares\n-----------\n\nMiddlewares allow for a data processing pipeline to pass data through.\n\n::\n\n      +-+  +-+  +-+\n      |M|  |M|  |M|\n      |I|  |I|  |I|\n      |D|  |D|  |D|\n      |D|  |D|  |D|\n    -\u003e+L+-\u003e+L+-\u003e+L+-\u003e\n      |E|  |E|  |E|\n      |W|  |W|  |W|\n      |A|  |A|  |A|\n      |R|  |R|  |R|\n      |E|  |E|  |E|\n      +-+  +-+  +-+\n\nWith this model, we gain a lot of flexibility in the behavior of our\nintegration. Middleware is up to the developer to create, and can be any\nof the following:\n\n-  Data post-processing, filtering, aggregation, or augmentation\n-  External integrations and interfaces\n-  Stimulate an API/web transaction from external actors or time-based\n   criteria\n-  Hooks and callbacks\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobotstudio%2Fbors","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobotstudio%2Fbors","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobotstudio%2Fbors/lists"}