{"id":20716306,"url":"https://github.com/waddafunk/tiny_mlops","last_synced_at":"2026-04-18T00:31:17.939Z","repository":{"id":230081339,"uuid":"778424319","full_name":"waddafunk/tiny_mlops","owner":"waddafunk","description":"Use your raspberry PI for free DevOps/MlOps infra for your private and public Github projects.","archived":false,"fork":false,"pushed_at":"2024-11-15T16:56:27.000Z","size":40,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-11T07:26:03.887Z","etag":null,"topics":["devops","mlflow","mlops","raspberry-pi","raspberry-pi-3","woodpecker"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/waddafunk.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-27T17:40:48.000Z","updated_at":"2024-11-15T16:56:31.000Z","dependencies_parsed_at":"2024-11-19T02:47:21.087Z","dependency_job_id":null,"html_url":"https://github.com/waddafunk/tiny_mlops","commit_stats":null,"previous_names":["waddafunk/tiny_mlops"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/waddafunk/tiny_mlops","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waddafunk%2Ftiny_mlops","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waddafunk%2Ftiny_mlops/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waddafunk%2Ftiny_mlops/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waddafunk%2Ftiny_mlops/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/waddafunk","download_url":"https://codeload.github.com/waddafunk/tiny_mlops/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/waddafunk%2Ftiny_mlops/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31951224,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-17T17:29:20.459Z","status":"ssl_error","status_checked_at":"2026-04-17T17:28:47.801Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["devops","mlflow","mlops","raspberry-pi","raspberry-pi-3","woodpecker"],"created_at":"2024-11-17T03:05:25.674Z","updated_at":"2026-04-18T00:31:17.915Z","avatar_url":"https://github.com/waddafunk.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Raspberry Pi MLOps Server\n\nUse your raspberry PI for **free DevOps/MlOps infra** for your private and public Github projects.\n\nA complete setup for running Woodpecker CI and MLflow on a Raspberry Pi, with secure tunneling and authentication.\n\n## Overview\n\nThis project sets up:\n- Woodpecker CI for continuous integration/deployment\n- MLflow for ML experiment tracking\n- Nginx reverse proxy with basic authentication\n- Bore tunnel for secure external access\n- Docker containers for all services\n\n## System Architecture\n\nBelow is a detailed diagram of all services and their interactions:\n\n```mermaid\ngraph TB\n    subgraph Internet\n        GH[GitHub]\n        Client[External Client]\n    end\n\n    subgraph \"Raspberry Pi\"\n        subgraph \"Docker Network: mlops-network\"\n            subgraph \"Authentication Layer\"\n                BT[Bore Tunnel\\nPort: host]\n                NG[Nginx Proxy\\nPort: 80]\n                Auth[/.htpasswd/]\n            end\n\n            subgraph \"CI/CD Services\"\n                WS[Woodpecker Server\\nPort: 8000/9000]\n                WA[Woodpecker Agent]\n                VD[/Docker Socket/]\n            end\n\n            subgraph \"ML Services\"\n                MF[MLflow Server\\nPort: 5000]\n                subgraph \"Persistent Storage\"\n                    ART[/Artifacts Directory/]\n                    DB[/SQLite Database/]\n                end\n            end\n        end\n    end\n\n    %% External Connections\n    Client --\u003e BT\n    BT --\u003e NG\n    GH \u003c--\u003e WS\n\n    %% Internal Connections\n    NG --\u003e WS\n    NG --\u003e MF\n    NG --- Auth\n    WS \u003c--\u003e WA\n    WA --- VD\n    MF --- ART\n    MF --- DB\n\n    %% Styling\n    classDef external fill:#f9f,stroke:#333,stroke-width:2px\n    classDef storage fill:#ff9,stroke:#333,stroke-width:2px\n    classDef service fill:#9ff,stroke:#333,stroke-width:2px\n    classDef auth fill:#f96,stroke:#333,stroke-width:2px\n\n    class GH,Client external\n    class Auth,VD,ART,DB storage\n    class WS,WA,MF service\n    class BT,NG auth\n```\n\n### Architecture Components\n\n1. **Authentication Layer**\n   - Bore Tunnel: Creates a secure public endpoint for external access\n   - Nginx Proxy: Handles routing and authentication\n   - .htpasswd: Stores basic auth credentials\n\n2. **CI/CD Services**\n   - Woodpecker Server: Manages CI/CD pipelines\n   - Woodpecker Agent: Executes CI/CD jobs\n   - Docker Socket: Allows agent to create containers\n\n3. **ML Services**\n   - MLflow Server: Tracks experiments and models\n   - Persistent Storage:\n     - Artifacts Directory: Stores ML model files\n     - SQLite Database: Stores experiment metadata\n\n4. **External Connections**\n   - GitHub: Integrates with Woodpecker for repositories\n   - External Clients: Access services through bore tunnel\n\n## Prerequisites\n\n- Raspberry Pi 3 or newer\n- Raspberry Pi OS (64-bit recommended)\n- Internet connection\n- GitHub account\n\n## Installation\n\n1. Clone this repository:\n   ```bash\n   git clone https://github.com/waddafunk/tiny_mlops.git\n   cd tiny_mlops\n   ```\n\n2. Make scripts executable:\n   ```bash\n   chmod +x *.sh\n   ```\n\n3. Run the installation script:\n   ```bash\n   ./install.sh\n   ```\n   This will:\n   - Install required packages (Docker, docker-compose, apache2-utils)\n   - Set up Docker permissions\n   - Create necessary directories\n   - Generate basic auth credentials\n   - Create initial environment files\n\n4. Set up GitHub OAuth:\n   - Go to GitHub Settings \u003e Developer Settings \u003e OAuth Apps\n   - Create a new OAuth app\n   - Wait for the bore tunnel URL from the next step before setting URLs\n\n5. Start the services:\n   ```bash\n   ./start_services.sh\n   ```\n   This script will:\n   - Generate new security secrets\n   - Start the bore tunnel\n   - Prompt for GitHub OAuth credentials\n   - Configure the environment with the bore tunnel URL\n   - Start all remaining services\n\n6. Complete GitHub OAuth setup:\n   - Return to your GitHub OAuth app settings\n   - Set Homepage URL to your bore tunnel URL (http://bore.pub:XXXXX)\n   - Set Authorization callback URL to http://bore.pub:XXXXX/authorize\n   - Save the changes\n\n## Services and Ports\n\n- Woodpecker CI: Port 8000 (externally accessible via bore tunnel)\n- MLflow: Port 5000 (locally accessible only)\n- Nginx: Port 80 (handles routing and authentication)\n- Bore Tunnel: Exposes services securely to the internet\n\n## Security Features\n\n- Basic authentication protects MLflow routes\n- Automated secret generation for Woodpecker\n- Secure cookie handling\n- GitHub OAuth integration\n- Local-only access for MLflow by default\n- All services run in isolated Docker containers\n\n## Configuration\n\n### Environment Variables\n\nThe `.env` file is automatically generated and contains:\n- `WOODPECKER_AGENT_SECRET`: Auto-generated secret for agent-server communication\n- `WOODPECKER_COOKIE_SECRET`: Auto-generated secret for cookie encryption\n- `WOODPECKER_HOST`: Automatically set to your bore tunnel URL\n- `WOODPECKER_GITHUB_CLIENT`: Your GitHub OAuth client ID\n- `WOODPECKER_GITHUB_SECRET`: Your GitHub OAuth client secret\n- `WOODPECKER_ADMIN`: Your GitHub username\n\n### Authentication\n\nBasic auth credentials are generated during installation. To manage them:\n\n```bash\n./scripts/setup_auth.sh\n```\n\nThis will either:\n- Show existing credentials\n- Generate new credentials if none exist\n- Provide options for credential management\n\n## Maintenance\n\n### Viewing Logs\n```bash\ncd services\ndocker-compose logs -f\n```\n\n### Restarting Services\n```bash\ncd services\ndocker-compose down\ndocker-compose up -d\n```\n\n### Updating Services\n```bash\ncd services\ndocker-compose pull\ndocker-compose up -d\n```\n\n### Managing the Bore Tunnel\nThe bore tunnel automatically reconnects if disconnected. To get the current URL:\n```bash\ndocker logs bore-tunnel\n```\n\n## Troubleshooting\n\n1. **Docker Permission Issues**\n   ```bash\n   sudo usermod -aG docker $USER\n   # Log out and log back in\n   ```\n\n2. **Service Access Issues**\n   - Check if services are running: `docker-compose ps`\n   - Verify bore tunnel URL: `docker logs bore-tunnel`\n   - Check service logs: `docker-compose logs [service-name]`\n\n3. **GitHub OAuth Issues**\n   - Verify callback URL matches bore tunnel exactly\n   - Check Woodpecker logs for OAuth errors\n   - Ensure GitHub OAuth credentials are correctly set in `.env`\n\n## File Structure\n```\n.\n├── config/\n│   └── nginx/\n│       ├── auth/\n│       │   └── .htpasswd\n│       └── default.conf\n├── mlflow/\n│   ├── artifacts/\n│   └── db/\n├── services/\n│   └── docker-compose.yml\n├── scripts/\n│   ├── setup_auth.sh\n│   └── mount_smb_share.sh\n├── install.sh\n├── start_services.sh\n└── .env\n```\n\n## Network Architecture\n\n```\nInternet \u003c-\u003e Bore Tunnel \u003c-\u003e Nginx Proxy \u003c-\u003e {Woodpecker CI, MLflow}\n```\n\n## Contributions\n\nContributions are welcome! Please submit pull requests for any improvements.\n\n## License\n\n                                 Apache License\n                           Version 2.0, January 2004\n                        http://www.apache.org/licenses/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwaddafunk%2Ftiny_mlops","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwaddafunk%2Ftiny_mlops","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwaddafunk%2Ftiny_mlops/lists"}