{"id":30290337,"url":"https://github.com/elkingarcia11/google-cloud-storage","last_synced_at":"2025-08-16T23:12:39.057Z","repository":{"id":304886994,"uuid":"1019676320","full_name":"elkingarcia11/google-cloud-storage","owner":"elkingarcia11","description":"A Python module for managing Google Cloud Storage operations using service account authentication.","archived":false,"fork":false,"pushed_at":"2025-07-21T03:20:52.000Z","size":12,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-21T05:26:55.913Z","etag":null,"topics":["google-cloud","google-cloud-platform","google-cloud-storage"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elkingarcia11.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-14T17:38:49.000Z","updated_at":"2025-07-21T03:20:56.000Z","dependencies_parsed_at":"2025-07-21T05:28:58.280Z","dependency_job_id":null,"html_url":"https://github.com/elkingarcia11/google-cloud-storage","commit_stats":null,"previous_names":["elkingarcia11/gcs-python-module","elkingarcia11/google-cloud-storage"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/elkingarcia11/google-cloud-storage","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elkingarcia11%2Fgoogle-cloud-storage","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elkingarcia11%2Fgoogle-cloud-storage/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elkingarcia11%2Fgoogle-cloud-storage/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elkingarcia11%2Fgoogle-cloud-storage/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elkingarcia11","download_url":"https://codeload.github.com/elkingarcia11/google-cloud-storage/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elkingarcia11%2Fgoogle-cloud-storage/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270781393,"owners_count":24643820,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-16T02:00:11.002Z","response_time":91,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["google-cloud","google-cloud-platform","google-cloud-storage"],"created_at":"2025-08-16T23:12:38.575Z","updated_at":"2025-08-16T23:12:39.047Z","avatar_url":"https://github.com/elkingarcia11.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Google Cloud Storage Python Module\n\nA comprehensive Python module for managing Google Cloud Storage operations using service account authentication.\n\n## Features\n\n- ✅ Create and delete buckets\n- ✅ List all buckets in a project\n- ✅ Upload files to buckets\n- ✅ Download files from buckets\n- ✅ List files in buckets\n- ✅ Delete files from buckets\n- ✅ Check bucket existence\n- ✅ Get file metadata\n- ✅ Service account authentication\n- ✅ Comprehensive error handling and logging\n- ✅ Environment variable support\n- ✅ Type hints for better development experience\n\n## Installation\n\n### 1. Clone this repository:\n\n```bash\ngit clone \u003crepository-url\u003e\ncd gcs-python-module\n```\n\n### 2. Create and activate virtual environment (recommended):\n\n```bash\n# Create virtual environment\npython -m venv venv\n\n# Activate virtual environment\n# On macOS/Linux:\nsource venv/bin/activate\n\n# On Windows:\n# venv\\Scripts\\activate\n```\n\n### 3. Install dependencies:\n\n```bash\n# Ensure pip is up to date\npip install --upgrade pip\n\n# Install dependencies\npip install -r requirements.txt\n```\n\n### 4. Set up Google Cloud service account:\n\n- Go to Google Cloud Console \u003e IAM \u0026 Admin \u003e Service Accounts\n- Create a new service account or select an existing one\n- Create a new key (JSON format)\n- Download the JSON key file to a secure location\n\n### 5. Configure environment variables:\n\n```bash\n# Set your project ID\nexport GOOGLE_CLOUD_PROJECT_ID=\"your-google-cloud-project-id\"\n\n# Set the service account credentials\nexport GOOGLE_APPLICATION_CREDENTIALS=\"/path/to/your/service-account-key.json\"\n```\n\n### 6. Add service account key to .gitignore:\n\n```bash\necho \"/path/to/your/service-account-key.json\" \u003e\u003e .gitignore\n```\n\n## Quick Start\n\n```python\nfrom gcs_client import GCSClient\n\n# Initialize client (uses GOOGLE_APPLICATION_CREDENTIALS environment variable)\ngcs = GCSClient()\n\n# List all buckets\nbuckets = gcs.list_buckets()\nprint(f\"Available buckets: {buckets}\")\n\n# Upload a file\nsuccess = gcs.upload_file(\"my-bucket\", \"local-file.txt\", \"remote-file.txt\")\nprint(f\"Upload: {'Success' if success else 'Failed'}\")\n\n# List files in bucket\nfiles = gcs.list_files(\"my-bucket\")\nprint(f\"Files in bucket: {files}\")\n\n# Download a file\nsuccess = gcs.download_file(\"my-bucket\", \"remote-file.txt\", \"downloaded-file.txt\")\nprint(f\"Download: {'Success' if success else 'Failed'}\")\n```\n\n## Usage Examples\n\n### Using the GCSClient Class\n\n```python\nfrom gcs_client import GCSClient\n\n# Initialize client (uses environment variables for authentication)\ngcs = GCSClient()\n\n# Create a bucket\n# (Requires Storage Admin permissions)\nsuccess = gcs.create_bucket(\"my-bucket\")\nprint(f\"Bucket creation: {'Success' if success else 'Failed'}\")\n\n# Upload a file\nsuccess = gcs.upload_file(\"my-bucket\", \"local-file.txt\", \"remote-file.txt\")\nprint(f\"Upload: {'Success' if success else 'Failed'}\")\n\n# Download a file\nsuccess = gcs.download_file(\"my-bucket\", \"remote-file.txt\", \"downloaded-file.txt\")\nprint(f\"Download: {'Success' if success else 'Failed'}\")\n\n# List files with prefix\nfiles = gcs.list_files(\"my-bucket\", prefix=\"data/\")\nprint(f\"Files with prefix 'data/': {files}\")\n\n# Get file metadata\nmetadata = gcs.get_file_metadata(\"my-bucket\", \"remote-file.txt\")\nif metadata:\n    print(f\"File size: {metadata['size']} bytes\")\n    print(f\"Content type: {metadata['content_type']}\")\n    print(f\"Created: {metadata['created']}\")\n\n# Check if bucket exists\nexists = gcs.bucket_exists(\"my-bucket\")\nprint(f\"Bucket exists: {exists}\")\n\n# Delete a file\nsuccess = gcs.delete_file(\"my-bucket\", \"remote-file.txt\")\nprint(f\"File deletion: {'Success' if success else 'Failed'}\")\n\n# Delete bucket (must be empty)\nsuccess = gcs.delete_bucket(\"my-bucket\")\nprint(f\"Bucket deletion: {'Success' if success else 'Failed'}\")\n```\n\n### Error Handling and Logging\n\n```python\nimport logging\n\n# Create client with debug logging\ngcs_debug = GCSClient(log_level=logging.DEBUG)\n\n# Create client with only warnings and errors\ngcs_quiet = GCSClient(log_level=logging.WARNING)\n\n# All methods return appropriate values on failure\nbuckets = gcs.list_buckets()  # Returns empty list if failed\nsuccess = gcs.upload_file(\"bucket\", \"file.txt\", \"remote.txt\")  # Returns False if failed\nmetadata = gcs.get_file_metadata(\"bucket\", \"file.txt\")  # Returns None if file not found\n```\n\n## API Reference\n\n### GCSClient Class\n\n#### Constructor\n\n```python\nGCSClient(log_level: int = logging.INFO)\n```\n\n#### Methods\n\n- `create_bucket(bucket_name: str) -\u003e bool` - Creates a new bucket\n- `delete_bucket(bucket_name: str) -\u003e bool` - Deletes a bucket (must be empty)\n- `list_buckets() -\u003e List[str]` - Lists all buckets in the project\n- `bucket_exists(bucket_name: str) -\u003e bool` - Checks if a bucket exists\n- `upload_file(bucket_name: str, source_file_name: str, destination_blob_name: str) -\u003e bool` - Uploads a file\n- `download_file(bucket_name: str, source_blob_name: str, destination_file_name: str) -\u003e bool` - Downloads a file\n- `list_files(bucket_name: str, prefix: str = \"\") -\u003e List[str]` - Lists files in a bucket\n- `delete_file(bucket_name: str, blob_name: str) -\u003e bool` - Deletes a file\n- `get_file_metadata(bucket_name: str, blob_name: str) -\u003e Optional[dict]` - Gets file metadata\n\n#### Return Values\n\n- **Boolean methods** (`create_bucket`, `delete_bucket`, `upload_file`, `download_file`, `delete_file`): Return `True` on success, `False` on failure\n- **List methods** (`list_buckets`, `list_files`): Return list of items on success, empty list `[]` on failure\n- **Metadata method** (`get_file_metadata`): Returns dictionary with file info on success, `None` if file not found\n- **Existence check** (`bucket_exists`): Returns `True` if bucket exists, `False` otherwise\n\n## Environment Variables\n\nSet the following environment variables:\n\n```bash\n# Google Cloud Project ID (REQUIRED)\nexport GOOGLE_CLOUD_PROJECT_ID=\"your-google-cloud-project-id\"\n\n# Path to your Google Cloud service account key file (REQUIRED)\n# Download this from Google Cloud Console \u003e IAM \u0026 Admin \u003e Service Accounts\nexport GOOGLE_APPLICATION_CREDENTIALS=\"/path/to/your/service-account-key.json\"\n```\n\n**Note**: The `GCSClient` uses service account authentication and requires both environment variables to be set.\n\n## Service Account Setup\n\n1. **Create a Service Account:**\n\n   - Go to Google Cloud Console\n   - Navigate to IAM \u0026 Admin \u003e Service Accounts\n   - Click \"Create Service Account\"\n   - Give it a name and description\n\n2. **Assign Permissions:**\n\n   - Add the following roles:\n     - Storage Admin (for full access)\n     - Storage Object Admin (for object operations only)\n     - Storage Object Viewer (for read-only access)\n\n3. **Create and Download Key:**\n\n   - Click on the service account\n   - Go to Keys tab\n   - Click \"Add Key\" \u003e \"Create new key\"\n   - Choose JSON format\n   - Download the key file\n\n4. **Set Environment Variable:**\n   ```bash\n   export GOOGLE_APPLICATION_CREDENTIALS=\"/path/to/your/service-account-key.json\"\n   ```\n\n## Testing\n\n### Running Integration Tests\n\nThe project includes comprehensive integration tests that interact with actual Google Cloud Storage buckets. See the [tests/README.md](tests/README.md) for detailed instructions on:\n\n- Setting up the test environment\n- Running individual tests\n- Understanding what each test does\n- Troubleshooting common issues\n\n### Quick Test Run\n\n```bash\n# Activate virtual environment\nsource venv/bin/activate\n\n# Set environment variables\nexport GOOGLE_CLOUD_PROJECT_ID=\"your-project-id\"\nexport GOOGLE_APPLICATION_CREDENTIALS=\"/path/to/service-account.json\"\n\n# Run all integration tests\npython -m pytest tests/integration_test.py -v\n\n# Run a specific test\npython -m pytest tests/integration_test.py::TestIntegrations::test_create_bucket -v\n```\n\n## Advanced Features\n\n### Logging\n\nEach GCSClient instance has its own logger with configurable log levels:\n\n```python\nimport logging\n\n# Create client with debug logging\ngcs_debug = GCSClient(log_level=logging.DEBUG)\n\n# Create client with only warnings and errors\ngcs_quiet = GCSClient(log_level=logging.WARNING)\n```\n\n### Error Handling\n\nAll methods return appropriate values on failure and log errors for debugging:\n\n- **Boolean methods**: Return `False` on failure\n- **List methods**: Return empty list `[]` on failure\n- **Metadata method**: Returns `None` if file not found\n- **All methods**: Log errors with detailed information\n\n### Permissions\n\nSome operations require specific permissions for your service account:\n\n- **Storage Admin**: For creating/deleting buckets\n- **Storage Object Admin**: For uploading/deleting files\n- **Storage Object Viewer**: For listing and downloading files\n\nIf your service account lacks permissions, the module will log warnings and return appropriate failure values.\n\n---\n\nFor more details, see the code and docstrings in `gcs_client.py`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felkingarcia11%2Fgoogle-cloud-storage","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felkingarcia11%2Fgoogle-cloud-storage","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felkingarcia11%2Fgoogle-cloud-storage/lists"}