{"id":37064436,"url":"https://github.com/thoth-station/selinon-worker","last_synced_at":"2026-01-14T07:31:45.733Z","repository":{"id":44645839,"uuid":"157422402","full_name":"thoth-station/selinon-worker","owner":"thoth-station","description":"Selinon worker used for data gathering, data cleansing and experiments in project Thoth","archived":true,"fork":false,"pushed_at":"2022-02-02T11:56:47.000Z","size":113,"stargazers_count":1,"open_issues_count":5,"forks_count":5,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-11-28T05:25:22.122Z","etag":null,"topics":["artificial-intelligence","thoth"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thoth-station.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-11-13T17:51:27.000Z","updated_at":"2023-01-28T02:53:59.000Z","dependencies_parsed_at":"2022-09-07T17:02:24.037Z","dependency_job_id":null,"html_url":"https://github.com/thoth-station/selinon-worker","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/thoth-station/selinon-worker","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Fselinon-worker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Fselinon-worker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Fselinon-worker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Fselinon-worker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thoth-station","download_url":"https://codeload.github.com/thoth-station/selinon-worker/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoth-station%2Fselinon-worker/sbom","scorecard":{"id":883027,"data":{"date":"2025-08-11","repo":{"name":"github.com/thoth-station/selinon-worker","commit":"a567578ecc5903684ea13a3571c14d3c5b738831"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.1,"checks":[{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":1,"reason":"Found 2/17 approved changesets -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Maintained","score":0,"reason":"project is archived","details":["Warn: Repository is archived."],"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: GNU General Public License v3.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 26 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":0,"reason":"53 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2023-120 / GHSA-45c4-8wx5-qw6w","Warn: Project is vulnerable to: PYSEC-2024-24 / GHSA-5h86-8mv2-jq9f","Warn: Project is vulnerable to: GHSA-5m98-qgg9-wh84","Warn: Project is vulnerable to: GHSA-7gpw-8wmc-pm8g","Warn: Project is vulnerable to: GHSA-8495-4g3g-x7pr","Warn: Project is vulnerable to: PYSEC-2024-26 / GHSA-8qpw-xqxj-h4r2","Warn: Project is vulnerable to: GHSA-9548-qrrj-x5pj","Warn: Project is vulnerable to: PYSEC-2023-246 / GHSA-gfw2-4jvh-wgfg","Warn: Project is vulnerable to: GHSA-pjjw-qhg8-p2p9","Warn: Project is vulnerable to: PYSEC-2023-250 / GHSA-q3qx-c6g2-7pw2","Warn: Project is vulnerable to: PYSEC-2023-251 / GHSA-qvrw-v9rv-5rjx","Warn: Project is vulnerable to: PYSEC-2021-76 / GHSA-v6wp-4m6f-gcjg","Warn: Project is vulnerable to: PYSEC-2023-247 / GHSA-xx9p-xxvh-7g8j","Warn: Project is vulnerable to: PYSEC-2022-42986 / GHSA-43fp-rhv2-5gv8","Warn: Project is vulnerable to: PYSEC-2023-135 / GHSA-xqr8-7jwr-rhp7","Warn: Project is vulnerable to: PYSEC-2024-60 / GHSA-jjg7-2v4v-x38h","Warn: Project is vulnerable to: GHSA-55x5-fj6c-h6m8","Warn: Project is vulnerable to: PYSEC-2021-19 / GHSA-jq4v-f5q6-mjqq","Warn: Project is vulnerable to: PYSEC-2020-62 / GHSA-pgww-xf46-h92r","Warn: Project is vulnerable to: PYSEC-2022-230 / GHSA-wrxv-2j5q-m38w","Warn: Project is vulnerable to: PYSEC-2021-356 / GHSA-2ww3-fxvq-293j","Warn: Project is vulnerable to: PYSEC-2024-167 / GHSA-cgvx-9447-vcch","Warn: Project is vulnerable to: PYSEC-2021-859 / GHSA-f8m6-h2c7-8h9x","Warn: Project is vulnerable to: PYSEC-2019-106 / GHSA-mr7p-25v2-35wr","Warn: Project is vulnerable to: PYSEC-2022-5 / GHSA-rqjh-jp2r-59cj","Warn: Project is vulnerable to: PYSEC-2021-142 / GHSA-8q59-q68h-6hv4","Warn: Project is vulnerable to: PYSEC-2018-49 / GHSA-rprw-h62v-c2w7","Warn: Project is vulnerable to: GHSA-9hjg-9r4m-mvj7","Warn: Project is vulnerable to: GHSA-9wx4-h78v-vm56","Warn: Project is vulnerable to: PYSEC-2023-74 / GHSA-j8r2-6x86-q33q","Warn: Project is vulnerable to: GHSA-29pr-6jr8-q5jm","Warn: Project is vulnerable to: GHSA-g92j-qhmh-64v2","Warn: Project is vulnerable to: GHSA-753j-mpmx-qq6g","Warn: Project is vulnerable to: GHSA-7cx3-6m66-7c5m","Warn: Project is vulnerable to: GHSA-8w49-h785-mj3c","Warn: Project is vulnerable to: PYSEC-2023-75 / GHSA-hj3f-6gcp-jg8j","Warn: Project is vulnerable to: GHSA-qppv-j76h-2rpx","Warn: Project is vulnerable to: GHSA-w235-7p84-xx57","Warn: Project is vulnerable to: GHSA-fh56-85cw-5pq6","Warn: Project is vulnerable to: GHSA-fm67-cv37-96ff","Warn: Project is vulnerable to: GHSA-wpqr-jcpx-745r","Warn: Project is vulnerable to: GHSA-34jh-p97f-mpxf","Warn: Project is vulnerable to: PYSEC-2023-212 / GHSA-g4mx-q9vg-27p4","Warn: Project is vulnerable to: PYSEC-2023-207 / GHSA-gwvm-45gx-3cf8","Warn: Project is vulnerable to: PYSEC-2019-133 / GHSA-mh33-7rrq-662w","Warn: Project is vulnerable to: GHSA-pq67-6m6q-mj2v","Warn: Project is vulnerable to: PYSEC-2019-132 / GHSA-r64q-w8jr-g9qp","Warn: Project is vulnerable to: PYSEC-2023-192 / GHSA-v845-jxx5-vc9f","Warn: Project is vulnerable to: PYSEC-2020-148 / GHSA-wqvq-5m8c-6g24","Warn: Project is vulnerable to: PYSEC-2021-108","Warn: Project is vulnerable to: PYSEC-2014-14 / GHSA-652x-xj99-gmcc","Warn: Project is vulnerable to: PYSEC-2014-13 / GHSA-cfj3-7x9c-4p3h","Warn: Project is vulnerable to: PYSEC-2018-28 / GHSA-x84v-xcm2-53pg"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-24T08:56:52.798Z","repository_id":44645839,"created_at":"2025-08-24T08:56:52.798Z","updated_at":"2025-08-24T08:56:52.798Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28413370,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T05:26:33.345Z","status":"ssl_error","status_checked_at":"2026-01-14T05:21:57.251Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","thoth"],"created_at":"2026-01-14T07:31:45.122Z","updated_at":"2026-01-14T07:31:45.722Z","avatar_url":"https://github.com/thoth-station.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"thoth-worker\n------------\n\nA worker implementation for running workflows - used for data gathering for\nproject Thoth.\n\nThe worker implementation is an implementation of Selinon's worker - see `Selinon \u003chttps://github.com/selinon/selinon\u003e`_ and `Selinon \u003chttps://selinon.readthedocs.io\u003e`_ docs for more info.\n\n\nVisualizing flows\n=================\n\nTo visualize available flows, visually show which storage adapters are used for which flows as well as how tasks are grouped, you can issue the following command that places SVG images into ``fig/`` directory.\n.. code-block::\n\n  pipenv run selinon-cli -vvv plot --nodes-definition thoth/worker/config/nodes.yaml --flow-definitions thoth/worker/config/flows --output-dir fig\n\nRunning thoth-worker in the cluster\n===================================\n\nThere is assigned a namespace to run thoth-worker and data aggregation part (ping someone from Thoth team on AICoE channel). The main interaction point is an `API service \u003chttps://github.com/thoth-station/selinon-api\u003e`_.\n\nRunning thoth-worker locally\n============================\n\nAll workflows are primarily designed to be run in the cluster, but Selinon offers a simple CLI interface to run flows locally.\n\nPlease configure AICoE gopass for passwords management (ping somebody on AICoE channel for more info), you can inject your passwords into your environment by running:\n\n.. code-block:: console\n\n  eval $(gopass show aicoe/thoth/ceph.sh)\n  export THOTH_CEPH_BUCKET_PREFIX=data/thoth/$USER/\n\nPlease export ``THOTH_CEPH_BUCKET_PREFIX`` with your username as shown above so you do not clash with data used in the cluster - this way you will work with your own copy of data.\n\nBefore running flows, install all the requirements:\n\n.. code-block:: console\n\n  pipenv install\n\nNow you are able to run Selinon flows, to run keywords gathering flow, issue the following command:\n\n.. code-block:: console\n\n\n  export PYTHONPATH=.\n  pipenv run selinon-cli -vvv execute --nodes-definition thoth/worker/config/nodes.yaml --flow-definitions thoth/worker/config/flows --flow-name keywords\n\n\nIf you would like to run only some of the tasks in defined flows, feel free to use selective flow runs (see `docs \u003chttps://selinon.readthedocs.io/en/latest/selective.html\u003e`_ for more info):\n\n.. code-block:: console\n\n  pipenv run selinon-cli -vvv execute --nodes-definition thoth/worker/config/nodes.yaml --flow-definitions thoth/worker/config/flows --flow-name keywords --selective-task-names StackOverflowKeywordsAggregationTask\n\nTo see all the available flows, reach out to ``flows`` section in the ``thoth/worker/config/nodes.yaml`` file. It has a descriptive information with listing of all the available flows and arguments that are requested (you can specify arguments for CLI run via ``--node-args``, do not forget to use ``-j`` for JSON arguments).\n\nIt can be also useful to set ``--sleep-time`` to 0 for selinon-cli, not to wait for scheduler to schedule flows in large flow runs.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthoth-station%2Fselinon-worker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthoth-station%2Fselinon-worker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthoth-station%2Fselinon-worker/lists"}