https://github.com/datacamp/dbconnect-python
Easily connect to all internal databases. Only for internal use.
https://github.com/datacamp/dbconnect-python
Last synced: about 1 year ago
JSON representation
Easily connect to all internal databases. Only for internal use.
- Host: GitHub
- URL: https://github.com/datacamp/dbconnect-python
- Owner: datacamp
- Created: 2017-11-21T12:07:23.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2021-11-24T17:31:14.000Z (over 4 years ago)
- Last Synced: 2025-03-26T19:12:18.403Z (about 1 year ago)
- Language: Python
- Homepage:
- Size: 14.6 KB
- Stars: 7
- Watchers: 7
- Forks: 5
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# DBConnect
### Why use this module?
- Make it easy to discover new databases at DataCamp
- Allow us to rotate DB credentials, without anyone having to update their scripts
- Make it easy to open the documentation of a database
- Allow fine-grained access permissions by using the AWS Key and Secrets for each client.
## Installation
Make sure to ask the IDE team for your AWS Key and Secret.
```bash
pip install awscli
aws configure
> AWS Access Key ID:
> AWS Secret Access Key:
> Default region name: us-east-1
> Default output format:
pip install git+https://github.com/datacamp/dbconnect-python
```
Given that this package connects to databases, you will most likely need to install the MySQL and/or PostgreSQL clients for your OS.
## How to use
**For security reasons, we only allow DB connections from our VPN. Make sure to be connected when using this module.**
Get a list of our databases you can connect to:
```python
import dbconnect as dbc
dbc.get_databases()
['main-app',
'teach-app',
'challenges-app',
'projects-app',
'mobile-app',
'messenger-app',
'projector-app',
'datachats',
'timetracker']
```
Connect to a database and execute pandas queries:
```python
import dbconnect as dbc
import pandas as pd
engine = dbc.create_connection(database = 'main-app') # The 'main-app' string you can find from dbc.get_databases()
pd.read_sql('SELECT id, email FROM users ORDER BY id ASC LIMIT 10', engine)
```
Open the documentation of a database
```python
dbc.get_docs(database="main-app")
Opening "https://github.com/datacamp/main-app/wiki/Database-Documentation"
```
## AWS Parameter Store Structure
| Parameter | Value |
| --------- | ----- |
| /dbconnect/dbnames | List of ``s, separated with comma. |
For every application:
| Parameter | Value |
| --------- | ----- |
| `/dbconnect//type` | Type of database (mysql, postgresql) |
| `/dbconnect//endpoint` | Endpoint of the replica |
| `/dbconnect//port` | Port of the replica |
| `/dbconnect//user` | Username of the connection |
| `/dbconnect//database` | Name of the database |
| `/dbconnect//docs` | URL of the database documentation |
| `/dbconnect//password` | Password of the database. **Make sure the `type` is set to `SecureString`** |
These paramters are managed by terraform. (Ask the IDE team)