{"id":46845485,"url":"https://github.com/miztizm/farmore","last_synced_at":"2026-03-10T14:32:13.385Z","repository":{"id":324738347,"uuid":"1097940276","full_name":"miztizm/farmore","owner":"miztizm","description":"Farmore is a comprehensive Python CLI tool for backing up GitHub repositories and their associated data. Clone repositories, export issues, download releases, backup wikis, and more — all with a single command.","archived":false,"fork":false,"pushed_at":"2025-11-27T09:07:41.000Z","size":308,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-11-29T14:26:14.716Z","etag":null,"topics":["clone","github","mirror","mirrored-repository","pull","repository"],"latest_commit_sha":null,"homepage":"https://buymeacoffee.com/miztizm","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/miztizm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-17T03:43:38.000Z","updated_at":"2025-11-29T10:08:01.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/miztizm/farmore","commit_stats":null,"previous_names":["miztizm/farmore"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/miztizm/farmore","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miztizm%2Ffarmore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miztizm%2Ffarmore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miztizm%2Ffarmore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miztizm%2Ffarmore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/miztizm","download_url":"https://codeload.github.com/miztizm/farmore/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miztizm%2Ffarmore/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30337215,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T12:41:07.687Z","status":"ssl_error","status_checked_at":"2026-03-10T12:41:06.728Z","response_time":106,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clone","github","mirror","mirrored-repository","pull","repository"],"created_at":"2026-03-10T14:32:08.095Z","updated_at":"2026-03-10T14:32:13.366Z","avatar_url":"https://github.com/miztizm.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🥔 Farmore\n\n\u003e _\"Mirror every repo you own — in one command.\"_\n\n**Farmore** is a comprehensive Python CLI tool for backing up GitHub repositories and their associated data. Clone repositories, export issues, download releases, backup wikis, and more — all with a single command.\n\n**Version:** 0.9.0\n**License:** MIT\n**Python:** 3.10+\n\n---\n\n## ✨ Features\n\n### 🔄 Repository Management\n- **Bulk backups** - Clone all repos for a user or organization in one command\n- **Single repo backups** - Backup individual repositories with the `repo` command\n- **Repository search** - Search GitHub and clone matching repositories by keyword\n- **Smart updates** - Automatically pulls updates for existing repositories\n- **Parallel processing** - Fast backups with configurable worker threads\n- **SSH/HTTPS support** - Tries SSH first, falls back to HTTPS with token\n- **Bare/Mirror clones** - Use `--bare` flag for true 1:1 repository backups\n- **Git LFS support** - Use `--lfs` flag for repositories with large files\n- **Name regex filtering** - Filter repositories by name pattern with `--name-regex`\n- **Incremental backups** - Track backup state with `--incremental` flag\n\n### 📊 Data Export\n- **Issues export** - Export all issues to JSON/YAML with optional comments\n- **Pull requests export** - Export PRs with metadata and comments\n- **Workflows backup** - Backup GitHub Actions workflows and run history\n- **Releases download** - Download releases with metadata and binary assets\n- **Wiki backup** - Clone repository wikis as git repositories\n- **Labels export** - Backup repository labels with colors\n- **Milestones export** - Backup milestone configuration and progress\n- **Gists backup** - Clone all your GitHub gists\n- **Attachments download** - Download images/files from issues and PRs\n\n### 🔐 Access \u0026 Security\n- **Private repository support** - Full access with GitHub Personal Access Tokens\n- **Organization repos** - Backup all organization repositories\n- **Starred \u0026 watched repos** - Mirror repositories you've starred or are watching\n- **Secrets export** - Export repository secret names (values are never exposed)\n- **Followers/Following** - Export user's social connections\n- **Webhooks export** - Backup webhook configurations\n\n### 🎯 Advanced Features\n- **Flexible filtering** - By visibility (public/private/all), forks, archived status\n- **Rate limit handling** - Automatic retries with exponential backoff\n- **Organized structure** - Clean directory organization separating code from data\n- **Cross-platform** - Works on Linux, macOS, and Windows\n- **Beautiful CLI** - Powered by Typer and Rich with progress bars\n- **GitHub Enterprise** - Support via `--github-host` flag\n- **Discussions backup** - Export repository discussions (GraphQL API)\n- **Projects backup** - Export Projects v2 data (GraphQL API)\n\n---\n\n## 📦 Installation\n\n### Requirements\n\n**Python Version:** 3.10 or higher is required.\n\n**Additional Requirements:**\n- Git installed and available in PATH\n- GitHub Personal Access Token (for private repos and higher rate limits)\n\n---\n\n### 🎯 From PyPI (Recommended)\n\nThe easiest way to install Farmore is from the Python Package Index (PyPI):\n\n```bash\npip install farmore\n```\n\nThis is the recommended method for end users. Once installed, the `farmore` command will be available globally.\n\n**Verify installation:**\n```bash\nfarmore --version\nfarmore --help\n```\n\n---\n\n### 🧪 From TestPyPI (Pre-release Testing)\n\nTo test pre-release versions before they're published to PyPI:\n\n```bash\npip install --index-url https://test.pypi.org/simple/ farmore\n```\n\n**When to use this:**\n- Testing new features before official release\n- Helping with beta testing\n- Verifying bug fixes in development versions\n\n**Note:** TestPyPI packages may not have all dependencies available. You might need to install dependencies from regular PyPI separately.\n\n---\n\n### 📥 From GitHub Releases (Specific Versions)\n\nDownload a specific version from the [GitHub Releases page](https://github.com/miztizm/farmore/releases):\n\n1. **Download the `.whl` file** from the release you want (e.g., `farmore-0.3.0-py3-none-any.whl`)\n2. **Install the downloaded file:**\n\n```bash\npip install farmore-0.3.0-py3-none-any.whl\n```\n\n**When to use this:**\n- You need a specific version\n- You want to verify package integrity\n- You're installing in an offline environment (download first, install later)\n\n**Alternative - Source distribution:**\n```bash\n# Download the .tar.gz file instead\npip install farmore-0.3.0.tar.gz\n```\n\n---\n\n### 🔧 From Source (Development)\n\nFor developers who want to contribute or modify the code:\n\n```bash\n# Clone the repository\ngit clone https://github.com/miztizm/farmore.git\ncd farmore\n\n# Install in editable mode (changes to code take effect immediately)\npip install -e .\n\n# Or install with development dependencies (recommended for contributors)\npip install -e \".[dev]\"\n```\n\n**Development dependencies include:**\n- `pytest` - Testing framework\n- `pytest-cov` - Code coverage\n- `ruff` - Linting and formatting\n- `mypy` - Type checking\n- Additional testing utilities\n\n**Verify installation:**\n```bash\nfarmore --version\nfarmore --help\n```\n\n---\n\n### 🔄 Upgrading\n\nTo upgrade to the latest version:\n\n```bash\n# From PyPI\npip install --upgrade farmore\n\n# From TestPyPI\npip install --upgrade --index-url https://test.pypi.org/simple/ farmore\n```\n\n---\n\n### 🗑️ Uninstalling\n\nTo remove Farmore:\n\n```bash\npip uninstall farmore\n```\n\n---\n\n## 🚀 Quick Start\n\n```bash\n# Backup all repos for a user\nfarmore user miztizm\n\n# Backup a single repository\nfarmore repo microsoft/vscode\n\n# Search and clone repositories by keyword\nfarmore search \"nuxt laravel\" --limit 10\n\n# Backup with issues and pull requests\nfarmore repo miztizm/hello-world --include-issues --include-pulls\n\n# Backup everything for a repository\nfarmore repo python/cpython --all\n\n# Backup all repos for an organization\nfarmore org github --include-issues --include-pulls\n\n# With authentication (recommended)\nexport GITHUB_TOKEN=ghp_your_token_here\nfarmore user miztizm\n```\n\n_\"They say privacy is dead. Prove them wrong. Use a token.\"_ — schema.cx\n\n---\n\n## 🔑 Authentication\n\nFarmore uses GitHub Personal Access Tokens (PAT) for authentication. Tokens provide:\n- Access to private repositories\n- Higher rate limits (5,000 vs 60 requests/hour)\n- Organization repository access\n\n### Creating a Token\n\n**⭐ Recommended: Use Classic Personal Access Token**\n\n1. **Create a Classic PAT:** https://github.com/settings/tokens\n   - Click **\"Tokens (classic)\"** → **\"Generate new token (classic)\"**\n   - Give it a name: `farmore-backup`\n   - Select scope: ✅ **`repo`** (required for private repositories)\n   - Optional: ✅ **`delete_repo`** (only if using `farmore delete` command)\n   - Click \"Generate token\" and **copy it immediately**\n\n2. **Set environment variable:**\n\n```bash\n# Linux/macOS\nexport GITHUB_TOKEN=ghp_your_token_here\n\n# Windows PowerShell\n$env:GITHUB_TOKEN=\"ghp_your_token_here\"\n\n# Or create a .env file in the project directory\necho \"GITHUB_TOKEN=ghp_your_token_here\" \u003e .env\n```\n\n### Why Classic PAT?\n- ✅ Simple setup (just check `repo` scope)\n- ✅ Works with all repository types (personal + organization)\n- ✅ Proven reliability\n- ✅ Broader API compatibility\n\n### Rate Limits\n\n| Mode | Requests/Hour | Use Case |\n|------|---------------|----------|\n| ❌ Unauthenticated | 60 | Small public repos only |\n| ✅ Authenticated | 5,000 | Production, private repos |\n\n### Security Best Practices\n\n- ✅ Use environment variables or `.env` files\n- ✅ Set token expiration (90 days recommended)\n- ✅ Use minimal required permissions\n- ❌ Never commit tokens to version control\n- ❌ Avoid `--token` flag (exposes in shell history)\n\n---\n\n## 📚 Commands\n\nFarmore provides **14 commands** organized into 4 categories:\n\n### 🔄 Repository Backup\n\n#### `farmore user \u003cusername\u003e`\nBackup all private and public repositories for a GitHub user.\n\n```bash\n# Backup all repos\nfarmore user miztizm\n\n# Backup with data exports\nfarmore user miztizm --include-issues --include-pulls\n\n# Filter by visibility\nfarmore user miztizm --visibility public\n\n# Dry run\nfarmore user miztizm --dry-run\n```\n\n**Key Options:** `--visibility`, `--include-forks`, `--include-archived`, `--include-issues`, `--include-pulls`, `--include-workflows`, `--include-releases`, `--include-wikis`, `--max-workers`, `--dry-run`\n\n#### `farmore org \u003corgname\u003e`\nBackup all repositories for an organization. Same options as `user`.\n\n#### `farmore repo \u003cowner\u003e/\u003crepo\u003e`\nBackup a single repository with optional data exports.\n\n```bash\n# Just clone\nfarmore repo microsoft/vscode\n\n# Clone + data\nfarmore repo microsoft/vscode --include-issues --include-pulls\n\n# Everything\nfarmore repo python/cpython --all\n```\n\n**Key Options:** `--include-issues`, `--include-pulls`, `--include-workflows`, `--include-releases`, `--include-wikis`, `--all`\n\n#### `farmore search \u003cquery\u003e` 🆕\nSearch GitHub repositories by keyword and clone matching results.\n\n```bash\n# Basic search - clone top 10 results\nfarmore search \"smsbomber\"\n\n# Search with filters\nfarmore search \"machine learning\" --language python --min-stars 1000 --limit 20\n\n# Search and auto-confirm (skip prompt)\nfarmore search \"react components\" --limit 5 --yes\n\n# Custom output directory\nfarmore search \"awesome-python\" --output-dir ./my-collections --limit 15\n\n# Sort by stars (descending)\nfarmore search \"cli tools\" --language go --sort stars --order desc --limit 25\n\n# Flat structure (no owner subdirectories)\nfarmore search \"react hooks\" --flat-structure --limit 10\n```\n\n**Key Options:**\n- `--limit` (1-100): Maximum repositories to clone (default: 10)\n- `--language`: Filter by programming language (e.g., \"python\", \"javascript\", \"go\")\n- `--min-stars`: Minimum number of stars required\n- `--sort`: Sort order - \"best-match\" (default), \"stars\", \"forks\", or \"updated\"\n- `--order`: Sort direction - \"desc\" (default) or \"asc\"\n- `--yes` / `-y`: Skip confirmation prompt\n- `--output-dir`: Custom output directory (default: `./search-results/\u003cquery\u003e/`)\n- `--flat-structure`: Clone repos directly without owner subdirectories\n- `--workers`: Number of parallel workers for cloning (default: 4)\n\n**Output Structure (Default):**\n```\nsearch-results/\n└── \u003csanitized-query\u003e/\n    ├── \u003cowner1\u003e/\n    │   └── \u003crepo1\u003e/\n    ├── \u003cowner2\u003e/\n    │   └── \u003crepo2\u003e/\n    └── \u003cowner3\u003e/\n        └── \u003crepo3\u003e/\n```\n\n**Output Structure (Flat - with `--flat-structure`):**\n```\nsearch-results/\n└── \u003csanitized-query\u003e/\n    ├── \u003crepo1\u003e/\n    ├── \u003crepo2\u003e/\n    └── \u003crepo3\u003e/\n```\n\n**Note:** When using `--flat-structure`, repositories with duplicate names will have their owner appended (e.g., `repo-owner`) to avoid conflicts.\n\n**Rate Limits:**\n- Search API: 30 requests/minute (authenticated users)\n- Unauthenticated: 10 requests/minute\n\n**Note:** The search command uses GitHub's search API with support for advanced search qualifiers. You can also use GitHub's native search syntax directly in the query (e.g., `\"language:python stars:\u003e1000\"`).\n\n---\n\n### 📊 Data Export\n\n#### `farmore issues \u003cowner\u003e/\u003crepo\u003e`\nExport issues to JSON/YAML.\n\n```bash\nfarmore issues microsoft/vscode\nfarmore issues miztizm/hello-world --state open --include-comments\n```\n\n**Options:** `--format [json|yaml]`, `--state [all|open|closed]`, `--include-comments`\n\n#### `farmore pulls \u003cowner\u003e/\u003crepo\u003e`\nExport pull requests to JSON/YAML.\n\n```bash\nfarmore pulls microsoft/vscode\nfarmore pulls miztizm/hello-world --state open --include-comments\n```\n\n**Options:** `--format [json|yaml]`, `--state [all|open|closed]`, `--include-comments`\n\n#### `farmore workflows \u003cowner\u003e/\u003crepo\u003e`\nBackup GitHub Actions workflows.\n\n```bash\nfarmore workflows microsoft/vscode\nfarmore workflows actions/checkout --include-runs\n```\n\n**Options:** `--include-runs`\n\n#### `farmore releases \u003cowner\u003e/\u003crepo\u003e`\nDownload releases and assets.\n\n```bash\nfarmore releases microsoft/vscode\nfarmore releases nodejs/node --download-assets\n```\n\n**Options:** `--download-assets`\n\n#### `farmore wiki \u003cowner\u003e/\u003crepo\u003e`\nClone repository wiki.\n\n```bash\nfarmore wiki python/cpython\n```\n\n---\n\n### 🔍 Profile \u0026 Discovery\n\n#### `farmore profile [username]`\nExport user profile.\n\n```bash\nfarmore profile              # Your profile\nfarmore profile miztizm      # Another user's profile\n```\n\n**Options:** `--format [json|yaml]`\n\n#### `farmore starred [username]`\nMirror starred repositories.\n\n```bash\nfarmore starred              # Your starred repos\nfarmore starred miztizm      # Another user's starred repos\n```\n\n#### `farmore watched [username]`\nMirror watched repositories.\n\n```bash\nfarmore watched\n```\n\n---\n\n### 🔐 Security \u0026 Management\n\n#### `farmore secrets \u003cowner\u003e/\u003crepo\u003e`\nExport repository secret names (values never exposed).\n\n```bash\nfarmore secrets miztizm/farmore\n```\n\n**Options:** `--format [json|yaml]`\n\n#### `farmore delete \u003cowner\u003e/\u003crepo\u003e`\nDelete a repository (requires confirmation).\n\n```bash\nfarmore delete miztizm/old-project\nfarmore delete miztizm/test-repo --force  # Skip confirmation\n```\n\n**⚠️ Warning:** Requires `delete_repo` scope in your GitHub token.\n\n---\n\n## 📂 Directory Structure\n\nFarmore organizes backups with a clean structure that separates code from metadata:\n\n```\nbackups/\n└── \u003cusername\u003e/\n    ├── profile.json                          # User profile\n    ├── repos/                                # Git repositories\n    │   ├── private/\u003cowner\u003e/\u003crepo\u003e/\n    │   ├── public/\u003cowner\u003e/\u003crepo\u003e/\n    │   ├── starred/\u003cowner\u003e/\u003crepo\u003e/\n    │   ├── watched/\u003cowner\u003e/\u003crepo\u003e/\n    │   ├── organizations/\u003cowner\u003e/\u003crepo\u003e/\n    │   └── forks/\u003cowner\u003e/\u003crepo\u003e/\n    └── data/                                 # Metadata\n        ├── issues/\u003cowner\u003e_\u003crepo\u003e_issues.json\n        ├── pulls/\u003cowner\u003e_\u003crepo\u003e_pulls.json\n        ├── workflows/\u003cowner\u003e_\u003crepo\u003e/\n        ├── releases/\u003cowner\u003e_\u003crepo\u003e/\n        ├── wikis/\u003cowner\u003e_\u003crepo\u003e.wiki/\n        └── secrets/\u003cowner\u003e_\u003crepo\u003e_secrets.json\n```\n\n**Example:**\n```bash\n# After: farmore user miztizm --include-issues --include-pulls\nbackups/miztizm/\n├── profile.json\n├── repos/\n│   ├── public/miztizm/farmore/\n│   └── private/miztizm/secret-project/\n└── data/\n    ├── issues/\n    │   ├── miztizm_farmore_issues.json\n    │   └── miztizm_secret-project_issues.json\n    └── pulls/\n        ├── miztizm_farmore_pulls.json\n        └── miztizm_secret-project_pulls.json\n```\n\n---\n\n## 🔧 How It Works\n\n1. **Discovery** - Uses GitHub API to find repositories\n2. **Filtering** - Applies visibility, fork, and archived filters\n3. **Parallel Processing** - Processes multiple repos simultaneously\n4. **Smart Cloning** - Tries SSH first, falls back to HTTPS\n5. **Progress Reporting** - Real-time progress with rich output\n6. **Summary** - Final statistics and error reporting\n\n**Key Features:**\n- **Incremental backups** - Updates existing repos, clones new ones\n- **Error resilience** - Continues even if individual repos fail\n- **Rate limit aware** - Handles GitHub API limits automatically\n- **Organized output** - Auto-categorizes by type\n\n---\n\n## 🛠️ Development\n\n### Setup\n\n```bash\ngit clone https://github.com/miztizm/farmore.git\ncd farmore\npython -m venv venv\nsource venv/bin/activate  # Windows: venv\\Scripts\\activate\npip install -e \".[dev]\"\n```\n\n### Testing\n\n```bash\npytest                                    # Run all tests\npytest --cov=farmore --cov-report=html   # With coverage\npytest tests/test_github_api.py          # Specific test\n```\n\n### Code Quality\n\n```bash\nblack farmore/        # Format\nruff check farmore/   # Lint\nmypy farmore/         # Type check\n```\n\n---\n\n## 📄 License\n\nMIT License - see [LICENSE](LICENSE) file for details.\n\n---\n\n## 🤝 Contributing\n\nContributions welcome! Please:\n1. Check existing issues and PRs\n2. Follow the existing code style\n3. Add tests for new features\n4. Update documentation\n\n---\n\n## 💬 Support\n\n- 🐛 For bugs, [**Issues**](https://github.com/miztizm/farmore/issues)\n- 📩 For questions, [**Email**](mailto:\u0026#103;\u0026#105;\u0026#116;\u0026#104;\u0026#117;\u0026#98;\u0026#64;\u0026#109;\u0026#105;\u0026#122;\u0026#116;\u0026#105;\u0026#122;\u0026#109;\u0026#46;\u0026#99;\u0026#111;\u0026#109;)\n\n---\n\n## 🌟 Acknowledgments\n\nBuilt with [Typer](https://typer.tiangolo.com/), [Rich](https://rich.readthedocs.io/), [Requests](https://requests.readthedocs.io/), and [PyYAML](https://pyyaml.org/).\n\n---\n\n_\"Control is an illusion. But backups? Those are real.\"_ — schema.cx\n\n**Made with 🥔 by miztizm**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiztizm%2Ffarmore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmiztizm%2Ffarmore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiztizm%2Ffarmore/lists"}