{"id":23683856,"url":"https://github.com/elmezianech/autoinventory","last_synced_at":"2026-04-28T12:06:51.413Z","repository":{"id":270085591,"uuid":"900155521","full_name":"elmezianech/AutoInventory","owner":"elmezianech","description":"This project is an end-to-end, fully automated warehouse management solution designed to tackle real-world inventory challenges in the FMCG sector. From real-time data ingestion and predictive analytics to interactive dashboards, this project combines cutting-edge technologies and an event-driven architecture to simulate a business-ready system.","archived":false,"fork":false,"pushed_at":"2024-12-28T09:32:12.000Z","size":63,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-19T21:14:19.605Z","etag":null,"topics":["automation","dashboard","data-analysis","data-engineering-pipeline","docker","etl","glue-job","inventory-management","kafka","kpis","lambda-functions","lstm","ml-pipeline","mlflow","power-bi","pytorch","redshift","s3","streamlit","warehouse-management"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elmezianech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-08T02:21:13.000Z","updated_at":"2024-12-28T09:32:16.000Z","dependencies_parsed_at":"2024-12-28T10:23:45.639Z","dependency_job_id":"c7c873e9-826e-4795-9688-8f4ee08d1f7a","html_url":"https://github.com/elmezianech/AutoInventory","commit_stats":null,"previous_names":["elmezianech/autoinventory"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elmezianech%2FAutoInventory","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elmezianech%2FAutoInventory/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elmezianech%2FAutoInventory/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elmezianech%2FAutoInventory/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elmezianech","download_url":"https://codeload.github.com/elmezianech/AutoInventory/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239735261,"owners_count":19688262,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","dashboard","data-analysis","data-engineering-pipeline","docker","etl","glue-job","inventory-management","kafka","kpis","lambda-functions","lstm","ml-pipeline","mlflow","power-bi","pytorch","redshift","s3","streamlit","warehouse-management"],"created_at":"2024-12-29T20:21:41.955Z","updated_at":"2025-09-17T09:40:13.228Z","avatar_url":"https://github.com/elmezianech.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 📦 AutoInventory\r\n**A Fully Automated, Intelligent Warehouse Management System**  \r\n\r\n---\r\n\r\n## **Project Description**  \r\nAutoInventory is an end-to-end, event-driven warehouse management solution that addresses real-world FMCG inventory challenges. The system integrates real-time data streaming, predictive analytics, and interactive dashboards to optimize inventory levels, prevent stockouts, and reduce waste—all with zero manual intervention.  \r\n\r\n---\r\n\r\n## **Dataset**  \r\nThis project uses a simulated FMCG dataset, enriched with:  \r\n- **Sales Metrics:** Sales volume, price, and promotion data.  \r\n- **Inventory Metrics:** Stock levels and replenishment lead times.  \r\n- **Temporal Components:** Date, weekday, and month.  \r\n- **Geospatial Information:** Store locations and product categories.  \r\n- **Dataset Link:** [FMCG Sales Demand Forecasting Dataset on Kaggle](https://www.kaggle.com/datasets/krishanukalita/fmcg-sales-demand-forecasting-and-optimization/data)  \r\n\r\n---\r\n\r\n## **Architecture and Technologies Used**  \r\n### **Architecture**  \r\n1. **Data Streaming:** Kafka (running in Docker) streams inventory data into S3.  \r\n2. **Data Laking:** AWS S3 acts as a centralized data lake, storing raw and processed data for further analysis.  \r\n3. **ETL Pipeline:** Glue processes data, and Lambda triggers transformations dynamically.  \r\n4. **Data Warehousing:** Redshift stores analytical-ready data.  \r\n5. **Dashboards:** Streamlit and Power BI provide interactive visualizations.  \r\n\r\n![image](https://github.com/user-attachments/assets/987af82b-4825-42f5-97f5-9b022a31140f)\r\n\r\n---\r\n\r\n## **Project Implementation**  \r\n### **Step 1: Real-Time Data Streaming**  \r\n- **Technology Used:** Apache Kafka (deployed with Docker)  \r\n- Simulates real-time inventory updates by streaming data from sales points to AWS S3 as a centralized data lake.  \r\n- **Key Features:**  \r\n  - Ensures idempotency with custom logic.  \r\n  - Handles hourly batching for efficient ingestion.\r\n\r\n### **Step 2: ETL Pipeline**  \r\n- **Technologies Used:** AWS Glue and Lambda  \r\n- **ETL Highlights:**  \r\n  - Extracts raw data from S3 and applies transformations like revenue, cost, and profit margin calculations.  \r\n  - Processes and loads transformed data into AWS Redshift for analytical queries.  \r\n- **Event-Driven:** Automatically triggered by S3 file uploads using Lambda.  \r\n\r\n### **Step 3: Predictive Analytics**  \r\n- **Technology Used:** PyTorch custom Neural Network Models  \r\n- Forecasts sales and stock levels 5–7 days into the future.  \r\n- **Key Features:**  \r\n  - Rolling forecasts for stability.  \r\n  - Advanced preprocessing with outlier resistance and missing data handling.  \r\n  - Automatic model retraining for evolving data patterns.  \r\n\r\n### **Step 4: Dashboards and Insights**  \r\n- **Technologies Used:** Streamlit and Power BI  \r\n- **Streamlit:** Provides forecasting, historical data line graphs, and basic insights in real time.\r\n![Screenshot 2024-12-27 181718](https://github.com/user-attachments/assets/fe7fa640-dc26-45ca-859a-f24912ace7a1)\r\n\r\n- **Power BI:** Offers interactive dashboards with drill-down capabilities by store location and product category, and advanced visuals for current stock levels, replenishment times, and revenue trends.  \r\n![warehouse_page-0001 (1)](https://github.com/user-attachments/assets/ed0e81f6-2601-43e3-a7e9-b1678532252e)\r\n--- \r\n\r\n### **Technologies**  \r\n- **Data Streaming:** Apache Kafka (Dockerized)  \r\n- **AWS S3:** Acts as a centralized data lake for raw and processed data.\r\n- **AWS Lambda:** For triggering ETL processes dynamically based on S3 events.\r\n- **ETL:** \r\n  - **AWS Glue:** For scalable data transformation and data loading to AWS Redshift.  \r\n- **Data Warehousing:** AWS Redshift  \r\n- **Machine Learning:** PyTorch (custom neural network models for time-series forecasting)  \r\n- **Dashboards:** Streamlit and Power BI  \r\n- **Cloud Management:** AWS IAM for permissions and AWS Secrets Manager for secure credential handling  \r\n- **Experiment Tracking:** MLflow  \r\n- **Database Interaction:** SQLAlchemy  \r\n- **Containerization:** Docker  \r\n\r\n---\r\n\r\n## **Business Impact**  \r\n- **Prevented Stockouts:** Predictive analytics keep shelves stocked with high-demand products.  \r\n- **Reduced Waste:** Optimized inventory minimizes spoilage and overstock.  \r\n- **Improved Decision-Making:** Automated KPIs like profit margins and replenishment times enable smarter choices.  \r\n\r\n---\r\n\r\n## **Connect with Me**  \r\nHave questions or want to collaborate? Let’s connect!  \r\n- **LinkedIn:** [Profile](https://www.linkedin.com/in/el-meziane-cha%C3%AFma/)  \r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felmezianech%2Fautoinventory","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felmezianech%2Fautoinventory","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felmezianech%2Fautoinventory/lists"}