{"id":28615967,"url":"https://github.com/steelcake/cherry-pipelines","last_synced_at":"2026-03-09T08:01:55.765Z","repository":{"id":293329090,"uuid":"981797234","full_name":"steelcake/cherry-pipelines","owner":"steelcake","description":"A collection of pipelines built with cherry","archived":false,"fork":false,"pushed_at":"2025-05-31T21:46:41.000Z","size":301,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-12T02:55:56.764Z","etag":null,"topics":["blockchain","clickhouse","data","pipeline","pyhton"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/steelcake.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-11T22:32:08.000Z","updated_at":"2025-05-31T21:45:46.000Z","dependencies_parsed_at":"2025-05-14T20:26:17.044Z","dependency_job_id":"fbadc2f6-16b9-496c-81a8-000fc286e10b","html_url":"https://github.com/steelcake/cherry-pipelines","commit_stats":null,"previous_names":["steelcake/cherry-pipelines"],"tags_count":21,"template":false,"template_full_name":null,"purl":"pkg:github/steelcake/cherry-pipelines","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steelcake%2Fcherry-pipelines","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steelcake%2Fcherry-pipelines/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steelcake%2Fcherry-pipelines/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steelcake%2Fcherry-pipelines/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/steelcake","download_url":"https://codeload.github.com/steelcake/cherry-pipelines/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steelcake%2Fcherry-pipelines/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30287446,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-09T02:57:19.223Z","status":"ssl_error","status_checked_at":"2026-03-09T02:56:26.373Z","response_time":61,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blockchain","clickhouse","data","pipeline","pyhton"],"created_at":"2025-06-12T02:45:20.161Z","updated_at":"2026-03-09T08:01:55.747Z","avatar_url":"https://github.com/steelcake.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cherry-pipelines\n\nThis is a collection of pipelines that are built using [cherry](https://github.com/steelcake/cherry).\n\nAll data is stored in ClickHouse.\n\n## Python version\n\nThis project is meant to be run with Python 3.12\n\nIf you are using `uv` for development it should pick this up automatically because of the `.python-version` in the project root.\n\nThe docker image is configured to use this version of Python as well.\n\n## Running a pipeline \n\nUse the `main` script to run a pipeline:\n\n```bash\nuv run scripts/main.py\n```\n\nIt takes these parameters as environment variables:\n\n- `CHERRY_PIPELINE_KIND`, \"evm\" or \"svm\".\n- `CHERRY_PIPELINE_NAME`, name of the pipeline to run e.g. \"erc20_transfers\".\n- `CHERRY_FROM_BLOCK`, specify the block that the indexing should start from. defaults to 0.\n- `CHERRY_TO_BLOCK`, specify the block that the indexing should stop at. has no default. Indexing waits for new blocks when it reaches the tip of the chain if this argument is left empty.\n- `CHERRY_EVM_PROVIDER_KIND`, specify which provider to use when indexing evm chains. Can be `hypersync` or `sqd`. Has no default and is required when indexing evm.\n- `CHERRY_EVM_CHAIN_ID`, specify the chain_id when indexing an evm chain. has no default and is required when indexing evm.\n- `CHERRY_PROVIDER_BUFFER_SIZE`, specify buffering between ingestion - processing - writer. Increasing this parameter might improve performance but can also cause higher memory usage. Defaults to 2.\n- `CHERRY_INIT_DB`, It runs db setup script instead of the pipeline script if this is set to \"true\". \n- `CLICKHOUSE_HOST`, defaults to `127.0.0.1`.\n- `CLICKHOUSE_PORT`, defaults to `8123`.\n- `CLICKHOUSE_USER`, defaults to `default`.\n- `CLICKHOUSE_PASSWORD`, defaults to empty string,\n- `RUST_LOG` as explained in [env-logger docs](https://docs.rs/env_logger/latest/env_logger/#enabling-logging)\n- `PY_LOG` as explained in [python logging docs](https://docs.python.org/3/howto/logging.html). Defaults to \"INFO\"\n\nAn `.env` file placed in the project root can be used to define these for development.\n\n## Running with docker\n\nWe publish a docker image that runs the `main` script.\n\n## Dev Setup\n\nRun the docker-compose file to start a clickhouse instance for development.\n\n```bash\ndocker-compose up -d\n```\n\nRun this to delete the data on disk:\n```bash\ndocker-compose down -v\n```\n\nAnd this to stop the container without deleting the data:\n```bash\ndocker-compose down\n```\n\n## Development\n\nThis repo uses `uv` for development.\n\n- Format the code with `uv run ruff format`\n- Lint the code with `uv run ruff check`\n- Run type checks with `uv run pyright`\n- Run the tests with `uv run pytest`\n\n## Data Provider\n\nAll svm pipelines use `SQD`.\n\nAll evm pipelines are configurable using the `CHERRY_EVM_PROVIDER_KIND` env variable.\n\n## License\n\nLicensed under either of\n\n * Apache License, Version 2.0\n   ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n * MIT license\n   ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\n\nat your option.\n\n## Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in the work by you, as defined in the Apache-2.0 license, shall be\ndual licensed as above, without any additional terms or conditions.\n\n## Sponsors\n\n[\u003cimg src=\"https://steelcake.com/envio-logo.png\" width=\"150px\" /\u003e](https://envio.dev)\n[\u003cimg src=\"https://steelcake.com/sqd-logo.png\" width=\"165px\" /\u003e](https://sqd.ai)\n[\u003cimg src=\"https://steelcake.com/space-operator-logo.webp\" height=\"75px\" /\u003e](https://linktr.ee/spaceoperator)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsteelcake%2Fcherry-pipelines","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsteelcake%2Fcherry-pipelines","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsteelcake%2Fcherry-pipelines/lists"}