https://github.com/alpha2phi/alphalib
Data engineering and data science library developed using Jupyter notebook through literate programming.
https://github.com/alpha2phi/alphalib
data-engineering data-science
Last synced: 3 months ago
JSON representation
Data engineering and data science library developed using Jupyter notebook through literate programming.
- Host: GitHub
- URL: https://github.com/alpha2phi/alphalib
- Owner: alpha2phi
- License: apache-2.0
- Created: 2020-12-07T00:07:41.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2021-02-05T12:48:43.000Z (over 5 years ago)
- Last Synced: 2025-12-15T09:13:30.784Z (7 months ago)
- Topics: data-engineering, data-science
- Language: Jupyter Notebook
- Homepage: http://alpha2phi.com/alphalib/
- Size: 1.04 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# Welcome to alphalib
> A library for your daily data engineering and data science routines.
This file will become your README and also the index of your documentation.
## Install
```bash
pip install alphalib
```
## How to use
### Ingest from Excel to `accounts` table in PostgreSQL
```python
# Ingest from Excel to `accounts` table in PostgreSQL
excel_source = file_sources.get(FileSource.Excel, file_path="data/accounts.xlsx")
config = {
'host': 'localhost',
'port': 5432,
'db': 'testdb',
'user': 'user1',
'password': 'userpwd'
}
pgsql_target = db_targets.get(DatabaseTarget.PostgreSQL, **config)
ingest(excel_source, pgsql_target, 'accounts')
```
2020-12-12 21:26:59,943 INFO(): {'user_id': INTEGER(), 'username': VARCHAR(length=50), 'password': VARCHAR(length=50), 'email': VARCHAR(length=255), 'created_on': TIMESTAMP(), 'last_login': TIMESTAMP()}
Total records in data/accounts.xlsx - 100
user_id - 100
username - 100
password - 100
email - 100
created_on - 1
last_login - 1
### Ingest from CSV to `accounts` table in MySQL
```python
# Ingest from CSV to `accounts` table in MySQL
csv_source = file_sources.get(FileSource.CSV, file_path="data/accounts.csv")
config = {
'host': 'localhost',
'port': 3306,
'db': 'testdb',
'user': 'user1',
'password': 'userpwd'
}
mysql_target = db_targets.get(DatabaseTarget.MySQL, **config)
ingest(csv_source, mysql_target, 'accounts')
```
2020-12-12 21:35:29,017 INFO(): {'user_id': INTEGER(), 'username': VARCHAR(length=50), 'password': VARCHAR(length=50), 'email': VARCHAR(length=255), 'created_on': TIMESTAMP(), 'last_login': TIMESTAMP()}
Total records in data/accounts.csv - 100
user_id - 100
username - 100
password - 100
email - 100
created_on - 1
last_login - 1