https://github.com/roboflow/roboflow_cloud_upload
How to upload images from Cloud Platforms (AWS, Azure, GCP) to Roboflow.
https://github.com/roboflow/roboflow_cloud_upload
Last synced: about 2 months ago
JSON representation
How to upload images from Cloud Platforms (AWS, Azure, GCP) to Roboflow.
- Host: GitHub
- URL: https://github.com/roboflow/roboflow_cloud_upload
- Owner: roboflow
- Created: 2023-10-26T20:51:11.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-10-26T23:28:01.000Z (over 2 years ago)
- Last Synced: 2025-02-25T09:59:40.591Z (over 1 year ago)
- Language: Python
- Size: 11.7 KB
- Stars: 0
- Watchers: 5
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Uploading Images to Roboflow using Presigned URLs
This repository contains a script for uploading images to a Roboflow project using presigned URLs. The script supports AWS S3, Azure Blob Storage, and Google Cloud Storage. Chec out the Examples folder for sample scripts uploading data to Roboflow using signed URLs for each of these cloud storage providers.
## Requirements
- Python 3.x
- boto3 for AWS S3
- azure-storage-blob for Azure
- google-cloud-storage for Google Cloud
- requests
## Getting Started
### AWS S3
Install the AWS SDK for Python (`boto3`):
```bash
pip install boto3
```
Code to generate a presigned URL:
```python
import boto3
from botocore.config import Config
def generate_presigned_url_aws(bucket_name, object_name, region='us-east-2'):
s3 = boto3.client('s3', region_name=region, config=Config(signature_version='s3v4'))
url = s3.generate_presigned_url('get_object',
Params={'Bucket': bucket_name, 'Key': object_name},
ExpiresIn=3600)
return url
```
### Azure Blob Storage
Install the Azure Blob Storage client library for Python:
```bash
pip install azure-storage-blob
```
Code to generate a signed URL:
```python
from azure.storage.blob import BlobServiceClient, generate_blob_sas, BlobSasPermissions
def generate_signed_url_azure(blob_service_client, container_name, blob_name):
blob_client = blob_service_client.get_blob_client(container=container_name, blob=blob_name)
sas_token = generate_blob_sas(blob_service_client.account_name,
container_name,
blob_name,
account_key=blob_service_client.credential.account_key,
permission=BlobSasPermissions(read=True),
expiry=datetime.utcnow() + timedelta(hours=1))
url = blob_client.url + '?' + sas_token
return url
```
### Google Cloud Storage
Install the Google Cloud Storage client library for Python:
```bash
pip install google-cloud-storage
```
Code to generate a signed URL:
```python
from google.cloud import storage
def generate_signed_url_gcp(bucket_name, blob_name):
storage_client = storage.Client()
bucket = storage_client.bucket(bucket_name)
blob = bucket.blob(blob_name)
url = blob.generate_signed_url(expiration=datetime.timedelta(hours=1), method='GET')
return url
```
## Example API Usage
Load the configuration, fetch object keys, and generate presigned URLs, then use `upload_to_roboflow()` to upload images. Save uploaded image IDs to avoid re-uploads.
```python
import requests
import urllib.parse
API_URL = "https://api.roboflow.com"
def upload_single_image_to_roboflow(api_key: str, project_name: str, signed_url: str, image_name: str, split: str = "train") -> bool:
"""
Uploads a single image to Roboflow using a signed URL.
Args:
api_key (str): Your Roboflow API key.
project_name (str): Your Roboflow project name.
signed_url (str): The signed URL for the image you want to upload.
image_name (str): The name you want to give to the image.
split (str): The data split you want to assign to the image (train/test/val).
Returns:
bool: True if upload was successful, False otherwise.
"""
# Construct the upload URL for Roboflow API
upload_url = "".join(
[
API_URL + "/dataset/" + project_name + "/upload",
"?api_key=" + api_key,
"&name=" + image_name,
"&split=" + split,
"&image=" + urllib.parse.quote_plus(signed_url)
]
)
# Make the POST request to upload the image
response = requests.post(upload_url)
if response.status_code == 200:
print(f"Successfully uploaded {image_name} to {project_name}")
return True
else:
print(f"Failed to upload {image_name}. Error: {response.content.decode('utf-8')}")
return False
# Example usage
api_key = "YOUR_API_KEY"
project_name = "YOUR_PROJECT_NAME"
signed_url = "YOUR_SIGNED_URL"
image_name = "example_image.jpg"
upload_single_image_to_roboflow(api_key, project_name, signed_url, image_name)
```
Replace the placeholders (`YOUR_API_KEY`, `YOUR_PROJECT_NAME`, `YOUR_SIGNED_URL`) with your actual values, then run the script to upload the image.