{"id":21131209,"url":"https://github.com/derak-isaack/pharmacysales_modelling","last_synced_at":"2026-04-20T10:33:25.799Z","repository":{"id":243565430,"uuid":"812769300","full_name":"derak-isaack/PharmacySales_Modelling","owner":"derak-isaack","description":"Apache Kafka Streamlit application for real-time tracking of pharmacy sales using the start schema data model.","archived":false,"fork":false,"pushed_at":"2024-06-29T21:56:59.000Z","size":485,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-14T20:09:10.686Z","etag":null,"topics":["apache-kafka","data-modeling","mysql-database","quix-streams","starschema","streamlit"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/derak-isaack.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-09T20:30:58.000Z","updated_at":"2024-06-29T21:57:02.000Z","dependencies_parsed_at":"2024-06-29T21:34:57.534Z","dependency_job_id":"93840490-7990-467c-83eb-5871b25c8006","html_url":"https://github.com/derak-isaack/PharmacySales_Modelling","commit_stats":null,"previous_names":["derak-isaack/pharmacysales_modelling"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/derak-isaack/PharmacySales_Modelling","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/derak-isaack%2FPharmacySales_Modelling","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/derak-isaack%2FPharmacySales_Modelling/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/derak-isaack%2FPharmacySales_Modelling/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/derak-isaack%2FPharmacySales_Modelling/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/derak-isaack","download_url":"https://codeload.github.com/derak-isaack/PharmacySales_Modelling/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/derak-isaack%2FPharmacySales_Modelling/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32043081,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-20T00:18:06.643Z","status":"online","status_checked_at":"2026-04-20T02:00:06.527Z","response_time":94,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-kafka","data-modeling","mysql-database","quix-streams","starschema","streamlit"],"created_at":"2024-11-20T05:50:54.570Z","updated_at":"2026-04-20T10:33:25.777Z","avatar_url":"https://github.com/derak-isaack.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![streamlit](https://img.shields.io/badge/Streamlit-FF4B4B?logo=streamlit\u0026logoColor=fff\u0026style=for-the-badge)\n![SQL](https://img.shields.io/badge/SQLAlchemy-D71F00?logo=sqlalchemy\u0026logoColor=fff\u0026style=for-the-badge)\n![SQLite](https://img.shields.io/badge/SQLite-003B57?logo=sqlite\u0026logoColor=fff\u0026style=for-the-badge)\n![Python](https://img.shields.io/badge/Python-3776AB?logo=python\u0026logoColor=fff\u0026style=for-the-badge)\n![Kafka](https://img.shields.io/badge/Apache%20Kafka-231F20?logo=apachekafka\u0026logoColor=fff\u0026style=for-the-badge)\n\n## \u003cdiv style=\"padding: 35px;color:white;margin:10;font-size:200%;text-align:center;display:fill;border-radius:10px;overflow:hidden;background-image: url(https://images.pexels.com/photos/7078619/pexels-photo-7078619.jpeg?auto=compress\u0026cs=tinysrgb\u0026w=1260\u0026h=750\u0026dpr=1)\"\u003e\u003cb\u003e\u003cspan style='color:black'\u003e\u003cstrong\u003e Pharmacy Sales Tracker \u003c/strong\u003e\u003c/span\u003e\u003c/b\u003e \u003c/div\u003e \n\n`Motivation:` With the ever rising need for automation and real-time tracking across sales organizations to minimize human error and identify fraud, I sought to develop a `Streamlit` application which uses `MySQL server` database and is intergrated with `Apache Kafka` which offers `Low latency` to ensures `real-time data streaming`.\n\n### \u003cdiv style=\"padding: 20px;color:white;margin:10;font-size:90%;text-align:left;display:fill;border-radius:10px;overflow:hidden;background-image: url(https://w0.peakpx.com/wallpaper/957/661/HD-wallpaper-white-marble-white-stone-texture-marble-stone-background-white-stone.jpg)\"\u003e\u003cb\u003e\u003cspan style='color:black'\u003e Project Overview\u003c/span\u003e\u003c/b\u003e \u003c/div\u003e\n\nStreamlit real-time Pharmacy sales tracker that uses the `star-schema` to track sales across several pharmacy outlets for a big pharma. The application leverages on using the `star-schema` which is:\n\n* Easier to understand and manage\n* Less dependant on table joins.\n* High performance.\n\nThe application also uses the `MySQL server database` for data entry which  has several advantages namely:\n\n* supports transactions.\n* Supports data integrity.\n* Handles severall transaction requests simultaneously.\n* Offers atomicity. \n\nThe application also intergates `Apache Kafka` for real-time data streaming as well as transformations. Using `Kafka` offers the following benefits namely:\n\n* `Data durability and reliability` because data is stored on disk across brokers\n* `Real-time data processing`\n* Flexibility in `batch and stream processing.`\n* `Data auditing and compliance`: With Change Data Capture (CDC) approaches, Kafka facilitates data replication across multiple systems or databases, ensuring accurate and consistent data for auditing and compliance purposes. \n\n### \u003cdiv style=\"padding: 20px;color:white;margin:10;font-size:90%;text-align:left;display:fill;border-radius:10px;overflow:hidden;background-image: url(https://w0.peakpx.com/wallpaper/957/661/HD-wallpaper-white-marble-white-stone-texture-marble-stone-background-white-stone.jpg)\"\u003e\u003cb\u003e\u003cspan style='color:black'\u003e Objectives \u0026 description\u003c/span\u003e\u003c/b\u003e \u003c/div\u003e\n\nDevelop a data model that follows the `star-schema` approach having the `dimensions` and `facts` table. The `table-models` can be found [here](pharmacy_sales_tracker.sql) which typically follows the `sql` approach. \n\nDefining the tables in a separate file offers a more flexible approach for the application suppose further change may arise. It also provides easy debugging for the application. \n\n`ERD-diagram` ![ERD](ERD_diagram.png)\n\nThis [python-file](helpers.py) defines a class using the traditional `python OOP` approach which offers more customization and flavour to the main `streamlit application`.It also allows form sharing from the `doctor table`, `Employee table` and `Drug items` tables which are the `dimension tables` which very vital in providing more context to the `Facts table`. \n\nIntergrate `Apache Kafka` into the streamlit application to serve as the `Producer`. The data should be in `JSON` formart for easier ingestion into the `Kafka topics`. This is made possible by using the `serializer` which allows for transformation of data into `JSON` formart. \n\nRead data from `Kafka topics` by a consumer to allow for `Real-time` data streaming as well as processing. The consumer can be found [here](kafka_consumer.py)\n\nTo get started with `Apache Kafka`, the `Zookeper` should be running. On windows, the command to run the `Zookeeper` is `.\\bin\\windows\\zookeeper-server-start.bat .\\config\\zookeeper.properties`. The `Kafka server` should also be running and is possible by uisng the command `.\\bin\\windows\\kafka-server-start.bat .\\config\\server.properties`. \n\nN/B: Apache Kafka should be correctly configured in the environment variables to allow port communication.\n\n### \u003cdiv style=\"padding: 20px;color:white;margin:10;font-size:90%;text-align:left;display:fill;border-radius:10px;overflow:hidden;background-image: url(https://w0.peakpx.com/wallpaper/957/661/HD-wallpaper-white-marble-white-stone-texture-marble-stone-background-white-stone.jpg)\"\u003e\u003cb\u003e\u003cspan style='color:black'\u003e Conclusion \u0026 Future steps\u003c/span\u003e\u003c/b\u003e \u003c/div\u003e\n\nThe `Sreamlit app` is deployed locally due to the constraints of the database being available locally and `Apache Kafka` port usage. Here is a snippet of the `User Interface` for inputting sales data to provide real-time tracking. \n\n![Dimensions-snippet](Dimensions.png)\n![Facts-snippet](Facts.png)\n\nAfter running the `Consumer`, here is a snapshot of how the data streams in from the streamlit application \n\n![Data-stream](data_stream.png)\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fderak-isaack%2Fpharmacysales_modelling","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fderak-isaack%2Fpharmacysales_modelling","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fderak-isaack%2Fpharmacysales_modelling/lists"}