https://github.com/arunnpuram/dynamicsqldataextractor

DynamicSQLDataExtractor is a Python tool that simplifies executing dynamic SQL queries on MySQL databases, leveraging AWS Secrets Manager for secure access. It allows users to select queries and date ranges via a CLI, processes the data, and saves it as CSVs for easy report generation.
https://github.com/arunnpuram/dynamicsqldataextractor

csv dask mysql-database python3 secrest

Last synced: 24 days ago
JSON representation

DynamicSQLDataExtractor is a Python tool that simplifies executing dynamic SQL queries on MySQL databases, leveraging AWS Secrets Manager for secure access. It allows users to select queries and date ranges via a CLI, processes the data, and saves it as CSVs for easy report generation.

• Host: GitHub
• URL: https://github.com/arunnpuram/dynamicsqldataextractor
• Owner: Arunnpuram
• Created: 2024-08-11T23:36:54.000Z (6 months ago)
• Default Branch: main
• Last Pushed: 2024-08-12T00:09:10.000Z (6 months ago)
• Last Synced: 2024-11-09T16:16:33.144Z (3 months ago)
• Topics: csv, dask, mysql-database, python3, secrest
• Language: Python
• Homepage:
• Size: 5.86 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        

# DynamicSQLDataExtractor

**DynamicSQLDataExtractor** is a Python-based tool designed to streamline the execution of dynamic SQL queries against MySQL databases. Leveraging AWS Secrets Manager for secure credential management, this tool enables users to securely access and query databases without exposing sensitive information. Through an intuitive command-line interface, users can select predefined queries and specify date ranges for data extraction. The extracted data is then efficiently processed and saved as CSV files in a designated directory, making it easy to generate custom reports based on the latest data.

## Table of Contents

- [Features](#features)

- [Getting Started](#getting-started)

  - [Prerequisites](#prerequisites)

  - [Installation](#installation)

  - [Configuration](#configuration)

- [Usage](#usage)

- [Contact](#contact)

## Features

- Securely manage database credentials using AWS Secrets Manager.

- Execute dynamic SQL queries against MySQL databases.

- Select queries and specify date ranges via a command-line interface.

- Process and save extracted data as CSV files for easy report generation.

## Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

### Prerequisites

- Python 3.x installed

- AWS CLI configured with appropriate permissions

- MySQL database accessible

### Installation

Clone the repository and install dependencies:

 - git clone https://github.com/yourusername/DynamicSQLDataExtractor.git 

 - cd DynamicSQLDataExtractor pip install -r requirements.txt

### Configuration

1. Store your MySQL credentials in AWS Secrets Manager.

2. Create a `queries.json` file with your SQL queries and descriptions.

## Usage

Run the tool from the command line:

-  python main.py --aws-profile your_aws_profile --secret-name your_secret_name

Follow the prompts to select a query and enter date ranges.

## Contact

My email- [email protected]