https://github.com/jpedrou/rain-prediction

Application made with the aim of predict when it will rain today or tomorrow in Australia using Machine Learning and a queue system.
https://github.com/jpedrou/rain-prediction

airflow data-science data-visualization machine-learning preprocessing python sqlite3 streamlit

Last synced: 10 months ago
JSON representation

Application made with the aim of predict when it will rain today or tomorrow in Australia using Machine Learning and a queue system.

• Host: GitHub
• URL: https://github.com/jpedrou/rain-prediction
• Owner: jpedrou
• License: mit
• Created: 2024-07-26T21:38:27.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-07-29T21:40:59.000Z (over 1 year ago)
• Last Synced: 2025-02-13T05:35:34.858Z (12 months ago)
• Topics: airflow, data-science, data-visualization, machine-learning, preprocessing, python, sqlite3, streamlit
• Language: Python
• Homepage: https://rainprediction.streamlit.app/
• Size: 50.7 MB
• Stars: 2
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          


Rain Prediction in Australia


**Objective**

Develop a system that predicts if it will rain today and tomorrow in Australia using machine learning. Additionally, implement an user interface and a queue system using Apache Airflow to continuously update the model whenever the dataset changes.

**Technologies** 

Python|Streamlit|Airflow|SQLite

---|----|----|----|

|||

## Project Components

1. **Data Collection and Preparation**

    - Use the weather dataset provided by Kaggle: [Rain in Australia](https://www.kaggle.com/datasets/jsphyg/weather-dataset-rattle-package).

    - Perform Exploratory Data Analysis (EDA).

    - Preprocess the data to handle missing values, encode categorical variables and more.

2. **Modeling**

    - Select appropriate machine learning algorithm.

    - Train and validate the models using historical weather data to find the best performing model.

3. **Prediction System**

    - Develop a Python application that takes daily weather features as input and predicts whether it will rain today and tomorrow.

    - Implement the prediction logic using the trained machine learning model.

4. **Queue System with Apache Airflow**

    - Set up Apache Airflow to automate the workflow of updating the model.

    - Create DAGs (Directed Acyclic Graphs) in Airflow to define tasks such as data ingestion, preprocessing, model training, and deployment.

    - Schedule the DAGs to run at regular intervals or when new data is available.