{"id":24192664,"url":"https://github.com/sayantikabanik/fp2","last_synced_at":"2026-03-06T21:37:07.002Z","repository":{"id":114901567,"uuid":"437466630","full_name":"sayantikabanik/FP2","owner":"sayantikabanik","description":"End-to-end AutoML based, price forecasting framework. All the tools and technologies used are open source.","archived":false,"fork":false,"pushed_at":"2022-02-10T10:16:28.000Z","size":9554,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-05-02T04:07:18.115Z","etag":null,"topics":["dagster","etl-pipeline","flask","heroku","mlflow","model-monitoring","modeling","pycaret"],"latest_commit_sha":null,"homepage":"https://price-map-fp2.herokuapp.com/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sayantikabanik.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2021-12-12T06:04:41.000Z","updated_at":"2022-02-16T10:15:24.000Z","dependencies_parsed_at":"2023-09-25T00:57:45.900Z","dependency_job_id":null,"html_url":"https://github.com/sayantikabanik/FP2","commit_stats":{"total_commits":86,"total_committers":4,"mean_commits":21.5,"dds":"0.22093023255813948","last_synced_commit":"03969f37508ef91bc6413b73c5e50120429c24a8"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayantikabanik%2FFP2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayantikabanik%2FFP2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayantikabanik%2FFP2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayantikabanik%2FFP2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sayantikabanik","download_url":"https://codeload.github.com/sayantikabanik/FP2/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233770290,"owners_count":18727553,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dagster","etl-pipeline","flask","heroku","mlflow","model-monitoring","modeling","pycaret"],"created_at":"2025-01-13T16:07:08.344Z","updated_at":"2025-09-21T16:31:39.652Z","avatar_url":"https://github.com/sayantikabanik.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Crop price forecasting\n\n### Motivation \n- 15 to 16 % of GDP is contributed by agriculture and its allied sectors\n- 55 to 60 of Indian households are depending on income generated by Agri sectors: it emphasizes how big is the base of issue.\n- Price of agricultural products, such as preservable and non-preservable are volatile throughout seasons. Unstable prices are heart-breaking for producers and painful for consumers Also, head-ache for serving governments: on the supply side, managing of input material \u0026 on procurement, ensuring of MSPs (Minimum supporting prices)\n\n### Possible solution\n - A better estimation of prices commodities on long (12 to 36 months) and short term \n (3 to 5 months) can direct farmers towards the cultivation of remunerative crops \n - In turn, the Price managing strategy will eases the burden on incentives provided by \n governments. Also helps in the planning of input material\n - Price forecasting and demand estimation help in planning input supplies (Either raw \n or ready-to-consume) efficiently. In turn, the availability of products at affordable prices\n to consumers. \n\n### Data sources\n- [Indian government data repository](data.gov.in)\n- [National Agriculture Market](https://agmarknet.gov.in/)\n- [DEPARTMENT OF CONSUMER AFFAIRS](https://fcainfoweb.nic.in/Reports/Report_Menu_Web.aspx)\n- [India enviroment Portal](http://www.indiaenvironmentportal.org.in/media/iep/infographics/Rainfall%20in%20India/112%20years%20of%20rainfall.html)\n\n\n\n### Useful Resources to read and implement\n- [GitHub CI/CD](https://resources.github.com/ci-cd/)\n- [PyTest](https://realpython.com/pytest-python-testing/)\n- [Env management using conda](https://towardsdatascience.com/manage-your-python-virtual-environment-with-conda-a0d2934d5195)\n- [Pre-Commit hooks](https://pre-commit.com/)\n\n### Installing miniconda/light version of anaconda \n- [Info + details](https://docs.conda.io/en/latest/miniconda.html)\n\n### Commands to create and use conda environment\nWe are pinning versions of the packages\n\u003cdetails\u003e\n \u003csummary\u003e \"conda list\" look and feel after you complete the below steps \u003c/summary\u003e\n \n ```python\n\n# Name Version Build Channel\nappdirs 1.4.4 pyh9f0ad1d_0 conda-forge\nattrs 21.4.0 pyhd8ed1ab_0 conda-forge\nbrotli 1.0.9 h3422bc3_6 conda-forge\nbrotli-bin 1.0.9 h3422bc3_6 conda-forge\nbrotlipy 0.7.0 py38hea4295b_1003 conda-forge\nca-certificates 2021.10.8 h4653dfc_0 conda-forge\ncertifi 2021.10.8 py38h10201cd_1 conda-forge\ncffi 1.15.0 py38hc67bbb8_0 conda-forge\ncfgv 3.3.1 pyhd8ed1ab_0 conda-forge\ncharset-normalizer 2.0.9 pyhd8ed1ab_0 conda-forge\ncryptography 36.0.1 py38h10d4710_0 conda-forge\ncycler 0.11.0 pyhd8ed1ab_0 conda-forge\ndistlib 0.3.4 pyhd8ed1ab_0 conda-forge\neditdistance-s 1.0.0 py38h1670459_2 conda-forge\nfilelock 3.4.2 pyhd8ed1ab_0 conda-forge\nfonttools 4.28.5 py38hea4295b_0 conda-forge\nfreetype 2.10.4 h17b34a0_1 conda-forge\nidentify 2.3.7 pyhd8ed1ab_0 conda-forge\nidna 3.1 pyhd3deb0d_0 conda-forge\niniconfig 1.1.1 pyh9f0ad1d_0 conda-forge\njbig 2.1 h3422bc3_2003 conda-forge\njoblib 1.1.0 pyhd8ed1ab_0 conda-forge\njpeg 9d h27ca646_0 conda-forge\nkiwisolver 1.3.2 py38h1670459_1 conda-forge\nlcms2 2.12 had6a04f_0 conda-forge\nlerc 3.0 hbdafb3b_0 conda-forge\nlibblas 3.9.0 12_osxarm64_openblas conda-forge\nlibbrotlicommon 1.0.9 h3422bc3_6 conda-forge\nlibbrotlidec 1.0.9 h3422bc3_6 conda-forge\nlibbrotlienc 1.0.9 h3422bc3_6 conda-forge\nlibcblas 3.9.0 12_osxarm64_openblas conda-forge\nlibcxx 12.0.1 h168391b_0 conda-forge\nlibdeflate 1.8 h3422bc3_0 conda-forge\nlibffi 3.4.2 h3422bc3_5 conda-forge\nlibgfortran 5.0.0.dev0 11_0_1_hf114ba7_23 conda-forge\nlibgfortran5 11.0.1.dev0 hf114ba7_23 conda-forge\nliblapack 3.9.0 12_osxarm64_openblas conda-forge\nlibopenblas 0.3.18 openmp_h5dd58f0_0 conda-forge\nlibpng 1.6.37 hf7e6567_2 conda-forge\nlibtiff 4.3.0 h74060c4_2 conda-forge\nlibwebp-base 1.2.1 h3422bc3_0 conda-forge\nlibzlib 1.2.11 hee7b306_1013 conda-forge\nllvm-openmp 12.0.1 hf3c4609_1 conda-forge\nlz4-c 1.9.3 hbdafb3b_1 conda-forge\nmatplotlib 3.5.1 py38h150bfb4_0 conda-forge\nmatplotlib-base 3.5.1 py38hb140015_0 conda-forge\nmore-itertools 8.12.0 pyhd8ed1ab_0 conda-forge\nmunkres 1.1.4 pyh9f0ad1d_0 conda-forge\nncurses 6.2 h9aa5885_4 conda-forge\nnodeenv 1.6.0 pyhd8ed1ab_0 conda-forge\nnumpy 1.21.5 py38hb29071a_0 conda-forge\nolefile 0.46 pyh9f0ad1d_1 conda-forge\nopenjpeg 2.4.0 h062765e_1 conda-forge\nopenssl 1.1.1l h3422bc3_0 conda-forge\npackaging 21.3 pyhd8ed1ab_0 conda-forge\npandas 1.3.5 py38h3777fb4_0 conda-forge\npatsy 0.5.2 pyhd8ed1ab_0 conda-forge\npillow 8.4.0 py38h02acf36_0 conda-forge\npip 21.3.1 pyhd8ed1ab_0 conda-forge\npluggy 1.0.0 py38h10201cd_2 conda-forge\npre-commit 2.16.0 py38h10201cd_0 conda-forge\npy 1.11.0 pyh6c4a22f_0 conda-forge\npycparser 2.21 pyhd8ed1ab_0 conda-forge\npyopenssl 21.0.0 pyhd8ed1ab_0 conda-forge\npyparsing 3.0.6 pyhd8ed1ab_0 conda-forge\npysocks 1.7.1 py38h10201cd_4 conda-forge\npytest 6.2.5 py38h10201cd_1 conda-forge\npython 3.8.12 hab31e5c_2_cpython conda-forge\npython-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge\npython_abi 3.8 2_cp38 conda-forge\npytz 2021.3 pyhd8ed1ab_0 conda-forge\npyyaml 6.0 py38hea4295b_3 conda-forge\nreadline 8.1 hedafd6a_0 conda-forge\nrequests 2.26.0 pyhd8ed1ab_1 conda-forge\nscikit-learn 1.0.2 py38h2cd4032_0 conda-forge\nscipy 1.7.3 py38hd0c9ec0_0 conda-forge\nseaborn 0.11.2 hd8ed1ab_0 conda-forge\nseaborn-base 0.11.2 pyhd8ed1ab_0 conda-forge\nsetuptools 60.1.1 py38h10201cd_0 conda-forge\nsix 1.16.0 pyh6c4a22f_0 conda-forge\nsqlite 3.37.0 h72a2b83_0 conda-forge\nstatsmodels 0.13.1 py38h691f20f_0 conda-forge\nthreadpoolctl 3.0.0 pyh8a188c0_0 conda-forge\ntk 8.6.11 he1e0b03_1 conda-forge\ntoml 0.10.2 pyhd8ed1ab_0 conda-forge\ntornado 6.1 py38hea4295b_2 conda-forge\nunicodedata2 14.0.0 py38hea4295b_0 conda-forge\nurllib3 1.26.7 pyhd8ed1ab_0 conda-forge\nvirtualenv 20.4.7 py38h10201cd_1 conda-forge\nwheel 0.37.1 pyhd8ed1ab_0 conda-forge\nxz 5.2.5 h642e427_1 conda-forge\nyaml 0.2.5 h642e427_0 conda-forge\nzlib 1.2.11 hee7b306_1013 conda-forge\nzstd 1.5.1 h861e0a7_0 conda-forge\n ```\n \n\u003c/details\u003e\n\n```shell\nconda env create --file environment.yml\n```\n```shell\nconda activate fp2\n```\n```shell\nconda list\n```\n```shell\nconda info\n```\n```shell\nconda deactivate\n```\n### Installing the package in local \n```shell\npip install -e .\n```\n\n### Basic flow how to make best use of the workflow\n- Clone the repo\n- Create the env using the above commands \n- Install the package \n- All the tests goes into the `tests` directory \n- Any test experiements eg- `pickle files generated from autoML` goes under `experiments` directory\n- All modelling and related details into `forecasting_framework`, create subdirectories as required \n- Under forecasting_framework there are three submodules `utils`, `model`, `data` \n - `utils` reusable code components\n - `model` all modelling aspects (python scripts are highly encouraged)\n - `data` pipepine and raw data\n\n### How to contribute to the repo\n- Create a separate branch for your usecase \n- Raise PR (dont commit to main under any circumstance)\n\n### Running the data pipline \n- `python pipeline.py` - returns the processed data in ~/data directory\n- `dagit -f pipeline.py` - Dagster UI\n\n### Running tests\nInstall pytest (it is not part of environment.yml/package)\nIt should be installed locally\n- `pip install pytest==6.2.5`\n- `pytest tests`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsayantikabanik%2Ffp2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsayantikabanik%2Ffp2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsayantikabanik%2Ffp2/lists"}