{"id":16549594,"url":"https://github.com/settinghead/simple-pipeline-demo","last_synced_at":"2026-06-01T03:32:08.809Z","repository":{"id":142195478,"uuid":"158259482","full_name":"settinghead/simple-pipeline-demo","owner":"settinghead","description":"This repository serves as an example repo for getting your existing code into Skafos","archived":false,"fork":false,"pushed_at":"2018-11-19T16:53:01.000Z","size":31,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-04T13:46:47.719Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/settinghead.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-19T16:47:15.000Z","updated_at":"2019-05-30T17:56:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"28c5452e-b190-43e2-920f-d2aa61cbf1c8","html_url":"https://github.com/settinghead/simple-pipeline-demo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/settinghead/simple-pipeline-demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/settinghead%2Fsimple-pipeline-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/settinghead%2Fsimple-pipeline-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/settinghead%2Fsimple-pipeline-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/settinghead%2Fsimple-pipeline-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/settinghead","download_url":"https://codeload.github.com/settinghead/simple-pipeline-demo/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/settinghead%2Fsimple-pipeline-demo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33759178,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-01T02:00:06.963Z","response_time":115,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T19:29:58.094Z","updated_at":"2026-06-01T03:32:08.793Z","avatar_url":"https://github.com/settinghead.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Simpe Pipeline Example\nThis is a repository that serves as an example for a simple job you might have to run on a daily/hourly/weekly basis. It generates a mock data set, supposed to be representing some email events that are coming in and some user types associated with each of those emails.\n\nWe run a simple job to aggregate this data and from there it can either be written to file, stored in a database, send a message to a Slack channel, etc.\n\n### Pre-requisites\n- [**Sign Up**](https://dashboard.metismachine.io/sign-up) for a Skafos Account\n- [**Install Skafos**](https://docs.metismachine.io/docs/installation)\n- Authenticate using `skafos auth`\n- A repository on Github with code that you'd like to run on Skafos. If you don't have a repository and want to use a pre-baked example, fork [**this repository**](https://github.com/skafos/simple-pipeline-demo/).\n\nThe remaining steps in this tutorial use the example repository. \n\n## Step 1 - Clone and Initialize your Repository\n```shell\ngit clone https://github.com/username/simple-pipeline-demo.git # clone your repo\ncd simple-pipeline-demo # go into your project directory\nskafos init # initialize your new skafos project\n```\n\nAfter you have run the `skafos init` command, your directory will contain a `metis.config.yml` file. You will need to modify this file, as described below, for Skafos to run your code. \n\n## Step 2- Set Up Jobs\nIn order for Skafos to understand your pipeline, you must tell it in what order you would like it run. \nThis can be done by: \n- Creating job ids using `skafos create job` (see below).\n- Adding the jobs to your **metis.config.yml** file.\n\n```shell\n# Get your project-token\ncat metis.config.yml # take a look at your current config file\n\n# Create a job for each of your scripts\nskafos create job data_ingest.py --project \u003cinsert-your-project-token-here\u003e\nskafos create job generate_report.py --project \u003cinsert-your-project-token-here\u003e\n\n# Add these jobs to your config file by editing metis.config.yml\n# vi metis.config.yml or use whatever your favorite text editor is \n\n```\n## Step 3 - Add your modules\nIn order for Skafos to run properly, you must tell it what modules your jobs require. This can be specified in the requirements.txt file. \n\nFor example, our workflow requires pandas, numpy, etc. so we must add those to our **requirements.txt**.\n\n```shell\n# either use your favorite text editor or\n# use the below commands\ntouch requirements.txt;\necho \"pandas==0.23.4\" \u003e\u003e requirements.txt;\necho \"numpy==1.15.1\" \u003e\u003e requirements.txt;\necho \"tabulate==0.8.2\" \u003e\u003e requirements.txt;\necho \"datetime==4.3\" \u003e\u003e requirements.txt;\n\n# commit our changes to our repo\ngit add .\ngit commit -m \"Create jobs in config and Add modules to requirements file\"\n```\n\n## Step 4 - Add Skafos to your GitHub repo\nSkafos can follow updates you make to your application and deploy whenever you've made updates to your application. To do this, you must [**add the Skafos app to your repo**](https://github.com/apps/skafos).\n\n![Adding Skafos to your GitHub repo](https://files.readme.io/86869d1-Screenshot_from_2018-11-07_16-29-58.png)\n\n## Step 5 - Push to Skafos\nSimply git push! To see your project running on Skafos, check out your [dashboard](https://dashboard.metismachine.io/)\n```shell\ngit push skafos\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsettinghead%2Fsimple-pipeline-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsettinghead%2Fsimple-pipeline-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsettinghead%2Fsimple-pipeline-demo/lists"}