Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/aymane-maghouti/real-time-computer-performance-dashboard
This project aims to capture, store, and visualize real-time system performance metrics through an end-to-end data pipeline. By leveraging Python, MySQL, SQL Server, and Power BI, we've created a comprehensive solution to enhance decision-making.
https://github.com/aymane-maghouti/real-time-computer-performance-dashboard
dashboard data-pipeline mysql-connector mysql-database powerbi pyodbc python sql-server-database
Last synced: about 2 months ago
JSON representation
This project aims to capture, store, and visualize real-time system performance metrics through an end-to-end data pipeline. By leveraging Python, MySQL, SQL Server, and Power BI, we've created a comprehensive solution to enhance decision-making.
- Host: GitHub
- URL: https://github.com/aymane-maghouti/real-time-computer-performance-dashboard
- Owner: aymane-maghouti
- Created: 2023-08-09T15:23:19.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2023-09-24T09:00:14.000Z (over 1 year ago)
- Last Synced: 2023-09-24T16:57:31.606Z (over 1 year ago)
- Topics: dashboard, data-pipeline, mysql-connector, mysql-database, powerbi, pyodbc, python, sql-server-database
- Language: Python
- Homepage: https://aymane-maghouti.github.io
- Size: 891 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Performance Monitoring and Data Visualization Project
Welcome to the Performance Monitoring and Data Visualization project! This project aims to capture, store, and visualize real-time system performance metrics through an end-to-end data pipeline. By leveraging Python, MySQL, SQL Server, and Power BI, we've created a comprehensive solution to enhance decision-making.
----
## Table of Contents
- [Introduction](#introduction)
- [Features](#features)
- [Technologies Used](#technologies-used)
- [Setup Instructions](#setup-instructions)
- [Usage](#usage)
- [Data Flow](#data-flow)
- [Dashboard Preview](#dashboard-preview)
- [Acknowledgments](#acknowledgments)
- [Contact](#contact)
----
## Introduction
System performance monitoring is crucial for maintaining operational efficiency and preemptively identifying potential issues. This project takes a data-driven approach to capture, store, and visualize performance metrics in real-time, providing insights for informed decision-making.
---
## Features
- Fetches real-time Performance Monitoring Data using Python scripts(using psutil library).
- Stores captured data securely in a MySQL database (using mysql-connector-python API).
- Establishes a data pipeline to transfer data from MySQL to SQL Server database(using pyodbc API).
- Connects SQL Server to Power BI for creating dynamic visualizations.
- Constructs an interactive real-time dashboard for monitoring system performance.
---
## Technologies Used
- Python
- MySQL
- SQL Server
- Power BI
---
## Setup Instructions
1. Clone the repository: `git clone https://github.com/aymane-maghouti/Real-time-Computer-Performance-Dashboard.git`
2. Install required dependencies by following the instructions in the `requirements.txt` file after navigating to the project folder:
`pip install -r requirements.txt`
3. Set up the MySQL and SQL Server databases according to the provided schema.(Execute the `database_installation.sql` script on both the MySQL and SQL Server databases.)
4. Configure the Python scripts with the necessary connection details.
**For MySQL:**
*host* = 'localhost'
*database* = 'system_Performance'
*user* = your username (root by default)
*password* = your password
*table_name* = 'performance'
**For SQL Server:**
*Driver*={SQL Server};
*Server*=your server name;
*Database*=System_Performance;
*Trusted_Connection*=yes;
5. Install Power BI and connect to the SQL Server database(Or you can just open the system_performance.pbix file with powerBI software).
---
## Usage
1. Run the Python script to fetch Performance Monitoring Data.(main.py script)
2. Data is automatically stored in the MySQL database.
3. The data pipeline transfers data from MySQL to SQL Server.
4. Open the Power BI file to access the real-time dashboard.
If you want to see how it's work, check the LinkedIn post where there is a video explaining how to use it, here it is the link of the post : Click here
---
## Data Flow
1. Python script fetches data →
2. Data stored in MySQL database →
3. Data transferred to SQL Server via pipeline →
4. Power BI accesses SQL Server and creates visualizations.
---
## Dashboard Preview
The final result of the project is a `real time dashboard`
---
## Acknowledgments
-We would like to express our gratitude to the following resources for their valuable contributions to this project:
The official documentation of `Python` language.
The documentation provided by `MySQL` and `SQL Server` for configuring and working with the database.
The `Power BI` documentation, which aided in creating the dashboard.
Feel free to customize the content and functionality of this Project according to your specific requirements.
---
## Contact
Aymane Maghouti
---