{"id":21091017,"url":"https://github.com/muhammedrahil/cloudspark","last_synced_at":"2025-05-16T13:31:44.104Z","repository":{"id":252394390,"uuid":"840311364","full_name":"muhammedrahil/cloudspark","owner":"muhammedrahil","description":"The CloudsPark package is designed to simplify the management of AWS S3 and Lambda services. Whether you're working on the frontend or backend, CloudSpark provides an intuitive interface to generate presigned URLs and handle file uploads seamlessly.","archived":false,"fork":false,"pushed_at":"2025-03-03T14:08:37.000Z","size":92,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-27T14:19:27.578Z","etag":null,"topics":["aws","aws-lambda","aws-s3","package","pip","pyhton3"],"latest_commit_sha":null,"homepage":"https://pypi.org/search/?q=cloudspark","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/muhammedrahil.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-09T12:25:33.000Z","updated_at":"2025-03-03T14:08:40.000Z","dependencies_parsed_at":"2024-11-20T00:31:54.363Z","dependency_job_id":null,"html_url":"https://github.com/muhammedrahil/cloudspark","commit_stats":null,"previous_names":["muhammedrahil/cloudspark"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muhammedrahil%2Fcloudspark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muhammedrahil%2Fcloudspark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muhammedrahil%2Fcloudspark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muhammedrahil%2Fcloudspark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/muhammedrahil","download_url":"https://codeload.github.com/muhammedrahil/cloudspark/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254538447,"owners_count":22087867,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-lambda","aws-s3","package","pip","pyhton3"],"created_at":"2024-11-19T21:42:41.149Z","updated_at":"2025-05-16T13:31:43.002Z","avatar_url":"https://github.com/muhammedrahil.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CloudSpark\n\n\n`cloudspark` is a Python package that provides a convenient way to manage AWS S3 buckets and generate presigned URLs using `boto3`. It supports bucket creation, CORS management, policy setting, and presigned URL generation.\n\n## Features\n\n- **S3 Management**: Effortlessly manage your S3 buckets, objects, and file uploads with built-in methods.\n\n- **Presigned URL Generation**: Generate secure presigned URLs for your S3 objects, enabling users to upload files directly from the frontend without exposing your credentials.\n\n- **Seamless Integration**: Designed to work smoothly with both frontend and backend applications, making file uploads and cloud function management more accessible.\n\n\nThis repository contains several components. For more details on each component, refer to the respective README files.\n\n- [git repository](https://github.com/muhammedrahil/cloudspark)\n- [Documentation](https://github.com/muhammedrahil/cloudspark/blob/main/README.md)\n- [Js-Implementation](https://github.com/muhammedrahil/cloudspark/blob/main/js-implementation.md)\n\n\n## Installation\n\nYou can install `cloudspark` via pip. Make sure you have `boto3` installed as well.\n\n```bash \npip install cloudspark \n```\n## Usage\n#### Importing the Library\n```python\nfrom cloudspark import S3Connection\n```\n#### Initializing the Connection\nCreate an instance of `S3Connection` by providing your AWS credentials and region name\n```python\ns3_conn = S3Connection(access_key='YOUR_ACCESS_KEY',\n                       secret_access_key='YOUR_SECRET_ACCESS_KEY',\n                       region_name='YOUR_REGION_NAME')\n```\n\n#### Connecting to a Bucket\nEstablish a connection to an S3 bucket: \n```python\ns3_bucketclient = s3_conn.connect(bucket_name='your-bucket-name')\n```\nreturns an S3 client instance. \n\n#### Returns the current S3 client instance \n```python\ns3_bucketclient = s3_conn.get_instance()\n```\nreturns an S3 client instance. \n\n#### Creating a bucket\nCreates an S3 bucket with the provided name.\n```python\ns3_client = s3_conn.connect()\ns3_bucketclient = s3_conn.create_s3bucket(bucket_name='new-bucket-name')\n```\n\n#### Set Bucket CORS\nSet the CORS configuration for the connected bucket\n\n```python\ns3_conn.set_bucket_cors()\n\n# CORSRules: A list of dictionaries containing CORS rules.\n# If None, a default CORS rule allowing all origins and methods is applied.\ncors_rules = [\n    {\n        'AllowedHeaders': ['*'],\n        'AllowedMethods': ['GET', 'POST'],\n        'AllowedOrigins': ['*'],\n        'ExposeHeaders': [],\n        'MaxAgeSeconds': 3000\n    }\n]\n\ns3_conn.set_bucket_cors(CORSRules=cors_rules)\n```\n#### Get Bucket CORS\nRetrieve the CORS configuration for the connected bucket\n```python\ncors_config = s3_conn.get_bucket_cors()\n```\n\n#### Delete Bucket CORS\nDelete the CORS configuration from the connected bucket\n```python\ns3_conn.delete_bucket_cors()\n```\n#### Set Bucket Policy\nSet or update the bucket policy for the connected bucket\n```python\n\ns3_conn.set_bucket_policy()\n\n# A JSON string or dictionary representing the bucket policy. \n# If None, a default public read policy is applied.\n\npolicy = {\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Sid\": \"PublicReadGetObject\",\n            \"Effect\": \"Allow\",\n            \"Principal\": \"*\",\n            \"Action\": \"s3:GetObject\",\n            \"Resource\": f\"arn:aws:s3:::{bucket_name}/*\"\n        }\n    ]\n}\ns3_conn.set_bucket_policy(bucket_policy=policy)\n```\n#### Policy Fields Explaintion\n\n#### 1. Version\n    Key: \"Version\"\n    Value: \"2012-10-17\"\n    Explanation:\n    This specifies the version of the policy language. \"2012-10-17\" is the latest version and is recommended for use.\n#### 2. Id\n    Key: \"Id\"\n    Value: \"Policy1719828879302\"\n    Explanation:\n    The Id is an optional identifier for the policy. It helps in identifying the policy, especially when dealing with multiple policies. The value here is just a unique identifier that you can use to track or refer to this policy.\n#### 3. Statement\n    Key: \"Statement\"\n    Value: [ ... ]\n    Explanation:\n    This is an array of statements that define what actions are allowed or denied. Each statement contains specific instructions for who can perform what actions on which resources.\n    Inside the Statement Array:\n    Sid\n\n        Key: \"Sid\"\n            Value: \"Stmt1719828876812\"\n            Explanation:\n            The Sid (Statement ID) is an optional identifier for the specific statement. It helps in identifying and managing individual statements within the policy.\n            Effect\n\n        Key: \"Effect\"\n            Value: \"Allow\"\n            Explanation:\n            The Effect determines whether the statement allows or denies access. Here, \"Allow\" means that the actions specified in the Action field are permitted. If it were \"Deny\", those actions would be explicitly forbidden.\n            Principal\n\n        Key: \"Principal\"\n            Value: \"*\"\n            Explanation:\n            The Principal specifies the entity that is allowed or denied the actions. \"*\" means any entity (i.e., any user, role, or service) is allowed to perform the specified actions. You could also restrict this to a specific AWS account or IAM user.\n            Action\n\n        Key: \"Action\"\n            Value: \"s3:GetObject\"\n            Explanation:\n            The Action specifies what actions are allowed or denied. Here, \"s3:GetObject\" means that the entities specified in the Principal field are allowed to retrieve objects from the S3 bucket. \"s3:GetObject\" is the action used for downloading files from S3.\n            Resource\n\n        Key: \"Resource\"\n            Value: \"arn:aws:s3:::{{bucket_name}}/*\"\n            Explanation:\n            The Resource specifies the particular AWS resource that the policy applies to. Here, the resource is specified as \"arn:aws:s3:::{{bucket_name}}/*\", which means all objects (files) within the {{bucket_name}} S3 bucket.\n            The arn:aws:s3:::{{bucket_name}}/* is an Amazon Resource Name (ARN) that uniquely identifies the S3 bucket and all its objects. The /* at the end means that the policy applies to all objects within the {{bucket_name}} bucket.\n\n#### Retrieves Bucket Policy\nRetrieves the bucket policy for the connected S3 bucket\n```python\npolicy = s3_conn.get_bucket_policy()\n```\n\n#### Delete Bucket Policy\nDelete the bucket policy from the connected bucket\n```python\ns3_conn.delete_bucket_policy()\n```\n\n#### List User Policies\nList inline policies for an IAM user\n```python\npolicies = s3_conn.list_user_policies(UserName='user-name')\n```\n#### Block or Allow Public Access\nBlock or allow public access to the bucket\n```python\ns3_conn.public_access(block=True)  # Block public access\ns3_conn.public_access(block=False) # Allow public access\n```\n\n#### Generate Presigned Create URL\nGenerate a presigned URL for creating an object\n\n```python\nresponce = s3_conn.presigned_create_url(\n    object_name='object_name',\n    params={'key': 'value'}, # Or params= None\n    fields={'field': 'value'}, # Or fields= None\n    conditions=[{'condition': 'value'}], # Or conditions= None\n    expiration=3600\n)\n```\n`object_name` : The name of the object to be created in the S3 bucket.\n\n`params`: (Optional) Additional request parameters to include in the presigned URL.\n\n`fields` : (Optional) Pre-filled form fields to include in the presigned URL.\n\n`conditions`: (Optional) Conditions to include in the presigned URL.\n\n`expiration` : (Optional) Time in seconds for which the presigned URL should remain valid. Default is 3600 seconds (1 hour).\n\n##### Output\n\n```python\n{\n  \"url\": \"https://{{bucket_name}}.s3.amazonaws.com/\",\n  \"fields\": {\n    \"key\": \"{{object_name.extention}}\",\n    \"x-amz-algorithm\": \"AWS4-HMAC-SHA256\",\n    \"x-amz-credential\": \"{{access_key}}/20240814/us-east-1/s3/aws4_request\",\n    \"x-amz-date\": \"20240814T092932Z\",\n    \"x-amz-security-token\": \"{{sts_token}}\",\n    \"policy\": \"{{policy}}\",\n    \"x-amz-signature\": \"{{signature}}\"\n  }\n}\n\n```\n\n#### Generate Presigned Delete URL\nGenerate a presigned URL for Delete an object\n\n```python\nresponce = s3_conn.presigned_delete_url(object_name='object_name', expiration=3600)\n```\n\n##### Output\n\n```python\n\"https://{{bucket_name}}.s3.amazonaws.com/hdfwf.png?X-Amz-Algorithm=AWS4-HMAC-SHA256\u0026X-Amz-Credential={{access_key}}%2F20240812%2Fap-south-1%2Fs3%2Faws4_request\u0026X-Amz-Date=20240812T151744Z\u0026X-Amz-Expires=4600\u0026X-Amz-SignedHeaders=host\u0026X-Amz-Signature=a680ee638e9cc27f2ec21cae36ad807cffb2fc74e1e91fa55709f25f324d1e22\"\n```\n\n#### Uploads a file to the connected S3 bucket.\n\n```python\nresponce = s3_conn.upload_object(file=file, key_name=\"object_name\")\n\n# example:\nwith open('file_name',\"rb\") as file_obj:\n    s3_conn.upload_object(file=file_obj, key_name=\"object_name\")\n```\n`file`: Bytes of the file to upload\n\n`key_name`: S3 object name (e.g., 'folder/filename.txt').\n\n#### Retrieves an object from the connected S3 bucket.\n\nreturn: The object metadata\n\n```python\nkey_object = s3_conn.get_object(key_name=\"object_name\")\n\n```\n\n#### Deletes an object from the connected S3 bucket\n\n```python\nkey_object = s3_conn.delete_object(key_name=\"object_name\")\n\n```\n\n#### Lists objects in the connected S3 bucket.\n\n```python\nkey_object = s3_conn.get_objects()\n```\n\n```python\nkey_object = s3_conn.get_objects(only_objects=True)\n```\n\n`only_objects`: If True, returns a list of object metadata (excluding keys).\n\n```python\nkey_object = s3_conn.get_objects(only_keys=True)\n```\n\n`only_keys`: If True, returns a list of object keys.\n\n#### Retrieve metadata of an object\n\nThe `head_object` method retrieves metadata for an object stored in an S3 bucket. This includes details such as the content type, last modified date, and content length, but does not return the object's actual data.\n\n```python\nkey_object = s3_conn.head_object(key_name=\"object_name\")\n```\n\n#### policy_decode Function\n\nThe `policy_decode` function is designed to decode a Base64-encoded AWS S3 policy string and return it as a formatted JSON string. This is useful for inspecting and validating S3 presigned URL policies.\n\nDecodes a Base64-encoded policy string and returns it as a formatted JSON string\n```python\n\npolicy_dict = s3_conn.policy_decode(policy_encoded=\"policy_encoded string\")\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuhammedrahil%2Fcloudspark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmuhammedrahil%2Fcloudspark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuhammedrahil%2Fcloudspark/lists"}