https://github.com/silvermullet/aws-python-utils
Simplify use of AWS resources in your code with aws-python-utils
https://github.com/silvermullet/aws-python-utils
aws python python3
Last synced: 6 months ago
JSON representation
Simplify use of AWS resources in your code with aws-python-utils
- Host: GitHub
- URL: https://github.com/silvermullet/aws-python-utils
- Owner: silvermullet
- License: mit
- Created: 2018-06-05T02:20:24.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2019-01-23T18:03:10.000Z (over 7 years ago)
- Last Synced: 2025-09-13T14:56:35.739Z (10 months ago)
- Topics: aws, python, python3
- Language: Python
- Homepage:
- Size: 23.4 KB
- Stars: 1
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# aws-python-utils
Simplify use of AWS resources in your code with aws-python-utils
### Install
```
pip install aws-python-util
```
### SecretManager Util
* Easy secret retrieval
##### get_secret()
Use ENVIRONMENT variable "AWS_SECRET_MANAGER_SECRET" or pass in secret_key name.
###### Example Usage
Via ENVIRONMENT variable
```python
from aws_python_utils import secretmanager
import os
os.environ["AWS_SECRET_MANAGER_SECRET"] = "mysecret"
secretmanager = secretmanager.AwsSecretManager()
mysecret = secretmanager.get_secret()
```
Or pass in secret_key name ..
```python
from aws_python_utils import secretmanager
import os
secretmanager = secretmanager.AwsSecretManager()
mysecret = secretmanager.get_secret(secret_key="mysecret")
```
### S3 Util
* Streams an s3 object directly into a pandas DataFrame to avoid writing to disk and then loading from disk
* Uploads a DataFrame directly to s3
###### Example Usage
```python
from aws_python_utils.s3 import AwsS3
from io import BytesIO
import pandas as pd
import numpy as np
s3 = AwsS3()
bucket,key = s3.get_bucket_and_key_from_s3_path("s3://my-bucket/mypath/to/object")
print("bucket = " + bucket) # my-bucket
print("key = " + key) # mypath/to/object
# download a tab separated file schema: id val1 val2
df = s3.download_s3_file(s3_path, header=0, sep='\t', index='id')
df2 = pd.DataFrame(np.random.randint(low=0, high=10, size=(5, 5)), columns=['a', 'b', 'c', 'd', 'e'])
io_buffer = BytesIO()
df2.to_csv(io_buffer, columns=['a', 'c', 'e'], sep='\t', index=False)
s3.upload_to_s3("s3://your-bucket/path/to/object.tsv", io_buffer)
```
### EC2 Util
* Ami cleaner to mop up old ami's
###### Example Usage
```python
from aws_python_utils.ec2 import AwsEC2
from datetime import datetime
ec2 = AwsEC2()
# keep latest 5 ami'
ec2.clean_images("my-service-ami-dev-*", num_to_keep=5, cutoff_date=datetime(2018, 8, 1), images_to_keep=['ami-keepmeid'])
```