{"id":49407930,"url":"https://github.com/lingfish/pydeb-s3","last_synced_at":"2026-05-26T09:00:49.893Z","repository":{"id":353888655,"uuid":"1219584251","full_name":"lingfish/pydeb-s3","owner":"lingfish","description":"Python tool to create and manage APT repositories on S3. Upload .deb packages, manage manifests, GPG sign releases. AWS S3, GCS, Cloudfront, MinIO etc compatible.","archived":false,"fork":false,"pushed_at":"2026-05-20T06:45:25.000Z","size":638,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-20T07:05:51.424Z","etag":null,"topics":["apt","apt-repository","boto3","cli","deb","debian","gpg-signing","package-manager","python","s3"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/pydeb-s3/","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lingfish.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-04-24T02:49:52.000Z","updated_at":"2026-05-20T06:45:30.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/lingfish/pydeb-s3","commit_stats":null,"previous_names":["lingfish/pydeb-s3"],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/lingfish/pydeb-s3","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lingfish%2Fpydeb-s3","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lingfish%2Fpydeb-s3/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lingfish%2Fpydeb-s3/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lingfish%2Fpydeb-s3/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lingfish","download_url":"https://codeload.github.com/lingfish/pydeb-s3/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lingfish%2Fpydeb-s3/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33512325,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T03:12:49.672Z","status":"ssl_error","status_checked_at":"2026-05-26T03:12:47.976Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apt","apt-repository","boto3","cli","deb","debian","gpg-signing","package-manager","python","s3"],"created_at":"2026-04-28T22:35:50.115Z","updated_at":"2026-05-26T09:00:49.884Z","avatar_url":"https://github.com/lingfish.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pydeb-s3\n\n[![PyPI version](https://img.shields.io/pypi/v/pydeb-s3.svg)](https://pypi.org/project/pydeb-s3/)\n[![License](https://img.shields.io/pypi/l/pydeb-s3.svg)](LICENSE)\n[![Python versions](https://img.shields.io/pypi/pyversions/pydeb-s3.svg)](https://pypi.org/project/pydeb-s3/)\n[![GitHub stars](https://img.shields.io/github/stars/lingfish/pydeb-s3.svg)](https://github.com/lingfish/pydeb-s3/stargazers)\n\n**pydeb-s3** is a Python port of [deb-s3](https://github.com/deb-s3/deb-s3), a simple utility to make creating and managing APT repositories on S3.\n\nMost existing guides on using S3 to host an APT repository have you using something like [reprepro](http://mirrorer.alioth.debian.org/) to generate the repository file structure, and then [s3cmd](http://s3tools.org/s3cmd) to sync the files to S3.\n\nThe annoying thing about this process is it requires you to maintain a local copy of the file tree for regenerating and syncing the next time. Personally, my process is to use one-off virtual machines with [Vagrant](http://vagrantup.com), script out the build process, and then would prefer to just upload the final `.deb` from my Mac.\n\nWith **pydeb-s3**, there is no need for this. pydeb-s3 features:\n\n- Downloads the existing package manifest and parses it.\n- Updates it with the new package, replacing the existing entry if already there or adding a new one if not.\n- Uploads the package itself, the Packages manifest, and the Packages.gz manifest. It will skip the uploading if the package is already there.\n- Updates the Release file with the new hashes and file sizes.\n\n## Updated Features\n\npydeb-s3 has been rewritten in Python with modern tooling and additional capabilities:\n\n- Parses `.deb` files using the official `python-debian` library\n- Updates package manifests, replacing existing entries or adding new ones\n- Uploads packages, Packages manifest, and compressed manifests (`.gz`, `.bz2`, `.xz`)\n- Updates Release file with new hashes and file sizes\n- **GPG signing** of Release files for secure APT repositories\n- **S3-compatible storage** support (AWS S3, Google Cloud Storage, MinIO, etc.)\n- **Concurrent operation locking** to prevent conflicting uploads\n- **Dry-run mode** for clean/verify operations\n- **Configurable timestamps** with `--timestamps/--no-timestamps` flag, auto-detects TTY for clean interactive output\n- Modern CLI with Typer, featuring help text and shell completion\n\n## Installation\n\nInstall via pip:\n\n```bash\n$ pip install pydeb-s3\n```\n\nFor isolated installation, use [pipx](https://pipx.pypa.io/):\n\n```bash\n$ pipx install pydeb-s3\n```\n\n## Quick Start\n\nUpload a package to S3:\n\n```bash\n$ pydeb-s3 upload --bucket my-bucket my-deb-package-1.0.0_amd64.deb\n```\n\nFor S3-compatible endpoints (e.g., Google Cloud Storage, MinIO):\n\n```bash\n$ pydeb-s3 upload --bucket my-bucket \\\n    --endpoint https://storage.googleapis.com \\\n    --checksum-when-required \\\n    --visibility nil \\\n    my-deb-package-1.0.0_amd64.deb\n```\n\n## Usage\n\npydeb-s3 provides the following commands:\n\n```bash\n$ pydeb-s3 --help\nUsage: pydeb-s3 [OPTIONS] COMMAND [ARGS]...\n\n  Easily create and manage an APT repository on S3\n\nOptions:\n  --quiet                         Only show errors\n  --debug                         Enable debug output\n  --timestamps / --no-timestamps  Enable/disable timestamps (auto-detects TTY\n                                  by default)\n  --install-completion            Install completion for the current shell.\n  --show-completion               Show completion for the current shell, to\n                                  copy it or customize the installation.\n  --help                          Show this message and exit.\n\nCommands:\n  upload   Upload the given files to a S3 bucket as an APT repository.\n  list     List packages in given codename, component, and optionally architecture.\n  show     Show information about a package.\n  exists   Check if a package exists in the repository.\n  copy     Copy a package to another codename and component.\n  delete   Remove a package from the repository.\n  verify   Verify that the files in the package manifests exist.\n  clean    Remove orphaned package files.\n```\n\nFor detailed options per command, run `pydeb-s3 \u003ccommand\u003e --help`.\n\n## Output Control\n\npydeb-s3 provides control over log output:\n\n- **Timestamps**: Use `--timestamps` to enable or `--no-timestamps` to disable timestamp prefixes\n- **Auto-detection**: By default, timestamps are automatically disabled when output is a terminal (TTY) for cleaner interactive use, and enabled when output is piped or redirected for logging purposes\n- **Log format**: When timestamps are enabled, loguru's default format is used (includes timestamp with milliseconds, level, and source location)\n\n## Common Command Examples\n\n### List packages\n```bash\n$ pydeb-s3 list --bucket my-bucket --codename stable\n```\n\n### Show package info\n```bash\n$ pydeb-s3 show mypackage --bucket my-bucket --version 1.0.0\n```\n\n### Check if package exists\n```bash\n$ pydeb-s3 exists mypackage --bucket my-bucket --version 1.0.0\n```\n\n### Copy package to another codename\n```bash\n$ pydeb-s3 copy mypackage --bucket my-bucket --to-codename jammy --to-component main\n```\n\n### Verify repository integrity\n```bash\n$ pydeb-s3 verify --bucket my-bucket --fix-manifests\n```\n\n### Clean orphaned packages (dry-run first!)\n```bash\n$ pydeb-s3 clean --bucket my-bucket --dry-run\n$ pydeb-s3 clean --bucket my-bucket  # Actually remove orphans\n```\n\n## Configuration\n\n### AWS Credentials\n\npydeb-s3 uses standard `boto3` credential resolution:\n\n1. **Command-line options**: `--access-key-id`, `--secret-access-key`, `--session-token`\n2. **Environment variables**: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION`\n3. **AWS config file**: `~/.aws/credentials` and `~/.aws/config`\n\n### S3 Bucket\n\nThe `--bucket` option is required for all commands. Use `--prefix` to add a path prefix to all S3 objects.\n\n### Visibility / ACL\n\nControl uploaded file permissions with `--visibility`:\n- `public` (default): public-read ACL\n- `private`: private ACL\n- `authenticated`: authenticated-read ACL\n- `nil`: No ACL (for S3-compatible storage that doesn't support ACLs)\n\n### GPG Signing\n\nSign Release files with `--sign \u003cKEY_ID\u003e`. You can specify multiple keys if needed (though repeatable `--sign` is limited by Typer version constraints).\n\n## Development\n\npydeb-s3 uses [hatch](https://hatch.pypa.io/latest/) for packaging and dependency management.\n\n## License\n\nMIT License - see LICENSE file for details.\n\n## Credits\n\n- Original [deb-s3](https://github.com/deb-s3/deb-s3) by [Ken Robertson](https://github.com/krobertson)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flingfish%2Fpydeb-s3","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flingfish%2Fpydeb-s3","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flingfish%2Fpydeb-s3/lists"}