{"id":21417373,"url":"https://github.com/wambugu71/tommorow.io_data","last_synced_at":"2026-01-03T06:32:46.964Z","repository":{"id":249541363,"uuid":"716756551","full_name":"wambugu71/tommorow.io_Data","owner":"wambugu71","description":"Fetch  data weekly using  ETL approach from tommorow.io weather  forecast data.","archived":false,"fork":false,"pushed_at":"2024-08-29T20:50:26.000Z","size":43,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-23T05:45:15.513Z","etag":null,"topics":["api-client","data-mining","data-science","etl-automation","etl-pipeline","gcp-automation-gitops","gcp-storage","google-cloud-platform","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wambugu71.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-09T20:06:18.000Z","updated_at":"2024-08-29T20:48:28.000Z","dependencies_parsed_at":"2024-07-21T18:40:35.880Z","dependency_job_id":null,"html_url":"https://github.com/wambugu71/tommorow.io_Data","commit_stats":null,"previous_names":["wambugu71/tommorow.io_data"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wambugu71%2Ftommorow.io_Data","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wambugu71%2Ftommorow.io_Data/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wambugu71%2Ftommorow.io_Data/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wambugu71%2Ftommorow.io_Data/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wambugu71","download_url":"https://codeload.github.com/wambugu71/tommorow.io_Data/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243918629,"owners_count":20368745,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-client","data-mining","data-science","etl-automation","etl-pipeline","gcp-automation-gitops","gcp-storage","google-cloud-platform","python"],"created_at":"2024-11-22T19:14:51.738Z","updated_at":"2026-01-03T06:32:46.903Z","avatar_url":"https://github.com/wambugu71.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tomorrow.io Data Fetcher\n\nThis repository contains a workflow and scripts to fetch weekly weather forecast data from Tomorrow.io for the Dedan Kimathi area in Nyeri County and store it in Google Cloud Platform (GCP) and Firebase databases. It  was  a  project  to fetch data  for   a  research in  center  of datascince  `Dkut`. The execution of these scripts is automated using GitHub workflows.\n\nIt  is  based on ETL approach  to efficiently fetch, transform and  load  the  weather  data collected.\n## Table of Contents\n\n- [Introduction](#introduction)\n- [Features](#features)\n- [Prerequisites](#prerequisites)\n- [Installation](#installation)\n- [Usage](#usage)\n- [Configuration](#configuration)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Introduction\n\nThis project automates the process of fetching weather forecast data from Tomorrow.io for the Dedan Kimathi area in Nyeri County on a weekly basis. The data is then uploaded to both Firebase and GCP databases. The automation is handled by GitHub workflows, ensuring that the data is fetched and stored without manual intervention and  without  extra  costs  of  running  the  script.\n\n## Features\n\n- Fetches weekly weather forecast data from Tomorrow.io.\n- Stores the fetched data in Firebase and GCP databases.\n- Automated execution using GitHub workflows.\n- Configurable to suit different needs and areas.\n\n## Prerequisites\n\nBefore you begin, ensure you have met the following requirements:\n\n- You have a Tomorrow.io API key.\n- You have a Google Cloud Platform account with access to Firestore or another GCP database.\n- You have a Firebase project set up.\n- You have GitHub Actions enabled for your repository.\n\n## Installation\n\n1. Clone the repository:\n\n    ```bash\n    git clone https://github.com/wambugu71/tommorow.io_Data\n    cd tommorow.io_Data\n    ```\n\n2. Install the required dependencies:\n\n    ```bash\n    pip install -r requirements.txt\n    ```\n\n3. Set up your cloud service  account  and  download  the credentials.\n   i.e the  `json` format.\n   In this  case  named\n   ```\n   dsail-misc-new.json\n   weather.json\n   ```\n   These  are  from my  project, __Note__ : Don't  opensource  your  cloud  credentials.\n## Usage\n\nThe scripts are designed to be executed automatically by GitHub workflows. However, you can also run them manually for testing purposes.\n\nTo fetch and upload data manually, run:\n\n```bash\npython maincp.py\n```\n\n## Configuration\n\nThe repository contains the following key files and directories:\n\n- `maincp.py`: The main script to fetch data from Tomorrow.io and upload it to Firebase and GCP.\n- `.github/workflows/`: Contains the GitHub workflow files for automating the execution of the scripts.\n- `requirements.txt`: Lists the Python dependencies required for the project.\n- `dsail-misc-new.json` for  your  service  account  credentials. Ensure you  have  enabled neecessary  permisions.\n### GitHub Workflow\n\nThe GitHub workflow is defined in `.github/workflows/main.yml or  ken.yml`. It is set to run the `maincp.py` script on a weekly basis. You can modify the schedule as needed.\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n1. Fork the Project\n2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to the Branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\n## License\n\nDistributed under the MIT License. See `LICENSE` for more information.\n\n---\n\nFeel free to modify the code further for your needs. If you encounter any issues or have any questions, please open an issue in the repository.\n\nHappy coding!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwambugu71%2Ftommorow.io_data","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwambugu71%2Ftommorow.io_data","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwambugu71%2Ftommorow.io_data/lists"}