https://github.com/coderjolly/employee-information-retrieval-system
An information retrieval system constructed using Flask to perform ETL operations while ingesting New York Payroll data and provide a frontend user interface for manipulating filters and queries.
https://github.com/coderjolly/employee-information-retrieval-system
etl flask mvc numpy pandas python sqlite3
Last synced: 3 months ago
JSON representation
An information retrieval system constructed using Flask to perform ETL operations while ingesting New York Payroll data and provide a frontend user interface for manipulating filters and queries.
- Host: GitHub
- URL: https://github.com/coderjolly/employee-information-retrieval-system
- Owner: coderjolly
- Created: 2024-02-06T11:45:47.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-04-06T04:57:38.000Z (about 1 year ago)
- Last Synced: 2025-02-02T02:11:54.513Z (5 months ago)
- Topics: etl, flask, mvc, numpy, pandas, python, sqlite3
- Language: Python
- Homepage:
- Size: 14.5 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# New-York Employee Information Retrieval System
A flask application that makes use of MVC architecture along with a front-end for rendering results. The application accepts various inputs from users, and returns the requested query or the information related to the query. The front-end is handlred via a html interface and the inputs from the user are converted in parameters for the query whilst maintaining the backend server to handle more queries as well.
This project aims to use various **ETL** techniques and manipulations for **cleaning**, **indexing**, **extracting** and **loading** the dataset for running various user defined queries on the dataset.
## Directory Structure
```
├── Archive/ <- Contains scripts related to archiving data or code.
├── ETL/ <- Contains scripts for the Extract, Transform, and Load (ETL) process.
├── controller/ <- Contains scripts for controlling application behavior.
│ ├── __init__.py <- Initialization script for the controller module.
│ └── controller.py <- Script for controlling application behavior.
├── database/ <- Contains database files.
├── models/ <- Contains scripts defining application models.
├── ss/ <- Contains screenshots or images related to the project.
├── static/ <- Contains static files for the application.
├── tests/ <- Contains test scripts.
├── views/ <- Contains view templates for the application.
├── README.md <- Top-level README file providing an overview of the project.
├── app.py <- Main application script.
└── requirements.txt <- File listing project dependencies.
```
## Architecture
Model-View-Controller, which is a software architectural pattern commonly used in designing and developing user interfaces for web applications. It separates the application into three interconnected components:
- Model: The Model represents the data and business logic of the application. It encapsulates the data and provides methods to manipulate that data.
- View: The View is responsible for presenting the user interface to the user. It displays the data from the model to the user and sends user actions to the controller for processing.
- Controller: The Controller acts as an intermediary between the Model and the View. It receives input from the user via the View, processes that input , and updates the View accordingly.
## Installation
Whenever dealing with python based applications, it is a very good practice to make a virtual environment and then deal with the process of installing libraries and dependencies as the installation then affects one particular environment. One can use any virtual environment to inilise this project by either using `venv` or `conda`.
Next, we need to make sure that our dependies are in order, so we will use the following command to run the requirements for this project. The command is as follows:
```python
pip install -r requirements.txt
```
After the successfull installation of the libraries and dependencies, we will proceed forward to run the application by running the server from `app.py`. The command is as follows:
```python
python3.8 app.py
```
Following this, a prompt will be generated asking to route the attention to `locahost:5000` which is the default server address for this application rendering a front-end to interact with the user.
## Preview
The application loads at the Index page that has the options to choose from like viewing the db, finding an employee through an id or borough index and seeing the top-earners.
#### Homepage
#### Employees List
#### Find Employees through ID
#### Find Employees through Borough Index
#### Find the Top-Earners year-wise
