Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/flyanakin/CountMoney
A simple low-cost finance data pipeline orchestration. All you need is just python & SQL.
https://github.com/flyanakin/CountMoney
airtable-api dagster dbt etl finance modern-data-stack orchestration postgresql python sql stock tushare workflow
Last synced: 3 months ago
JSON representation
A simple low-cost finance data pipeline orchestration. All you need is just python & SQL.
- Host: GitHub
- URL: https://github.com/flyanakin/CountMoney
- Owner: flyanakin
- Created: 2022-12-28T02:45:09.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2023-04-26T02:58:22.000Z (over 1 year ago)
- Last Synced: 2024-07-02T19:36:31.421Z (4 months ago)
- Topics: airtable-api, dagster, dbt, etl, finance, modern-data-stack, orchestration, postgresql, python, sql, stock, tushare, workflow
- Language: Python
- Homepage:
- Size: 3.81 MB
- Stars: 13
- Watchers: 3
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# CountMoney
## Overview
A simple low-cost finance data pipeline orchestration. All you need is just python & SQL.
I compose the system of the following components, most of which are modern data stack components, meaning that the system is flexible and plug-and-play. You can switch to other components that you are familiar with. For example, in the EL part, you can switch to Airbyte or Fivetrans, etc. You can also write Python programs to transform your data. But note Dagster is the scheduling orchestration tool for this system. It is the foundation of this system. And you cannot replace it.
* datasources
* Finance data: [Tushare](https://tushare.pro/document/1) , a low-cost financial database, covering most Chinese stock market. If you purchase ¥500/yearly plan, you can use all api.
* Other data: [Airtable](https://airtable.com/), store some sensitive data , such as portfolio, costs, etc.
* extract & load: Ingestion as code, just write assets in orchestration code, and load data to postgres. IO logic is in ./resources/xxx_io_manager.py
* orchestration: Dagster cloud.
* Transformation: DBT.
* BI: Metadata, running on docker.
* Subcription
* wxwork webhook
* email
## Features
### Pipeline Management
* Orchestrating daily job by software-defined asset.
* Manage assets.
* Launch a run whenever another job materializes a specific asset.
### Visualization
### Subscription
## Models
* stock picks: all stock list with defence metrics and growth metrics
* performance: choose stocks with high growth from the stocks that released the performance forecast
* preview: choose stocks with high growth from the stocks that released the preview