{"id":25649789,"url":"https://github.com/openchami/quack","last_synced_at":"2025-10-27T21:09:04.761Z","repository":{"id":252466726,"uuid":"840530651","full_name":"OpenCHAMI/quack","owner":"OpenCHAMI","description":"A DuckDB and Parquet local database","archived":false,"fork":false,"pushed_at":"2024-08-10T00:00:31.000Z","size":9,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-09-16T05:58:02.565Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OpenCHAMI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-09T23:43:53.000Z","updated_at":"2024-08-10T00:00:34.000Z","dependencies_parsed_at":"2024-08-10T00:47:23.894Z","dependency_job_id":"d72dc2a3-1b59-4124-8ab8-6ca5dbb1cabd","html_url":"https://github.com/OpenCHAMI/quack","commit_stats":null,"previous_names":["openchami/quack"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenCHAMI%2Fquack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenCHAMI%2Fquack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenCHAMI%2Fquack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenCHAMI%2Fquack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OpenCHAMI","download_url":"https://codeload.github.com/OpenCHAMI/quack/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240329482,"owners_count":19784456,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-23T14:33:53.680Z","updated_at":"2025-10-27T21:09:04.741Z","avatar_url":"https://github.com/OpenCHAMI.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# QuackQuack\n\nQuackQuack is a resilient Go library for managing DuckDB databases with support for periodic snapshots, restoration, and comprehensive health monitoring. It provides robust error handling, graceful degradation, and detailed status reporting for production environments.\n\n## Features\n\n- 🔄 **Automatic Snapshots**: Periodic database snapshots in Parquet format\n- 🔧 **Resilient Extension Loading**: Multiple fallback strategies for DuckDB extensions\n- 🏥 **Health Monitoring**: Comprehensive health checks and status reporting\n- ⚡ **Graceful Degradation**: Continues operation even when optional features fail\n- 🛡️ **Input Validation**: Early detection of configuration issues\n- 📊 **Detailed Logging**: Rich logging and error reporting\n- 🔍 **Extension Status Tracking**: Monitor which extensions loaded successfully\n\n## Installation\n\n```sh\ngo get github.com/OpenCHAMI/quack/quack\n```\n\n## Quick Start\n\n### Basic Usage\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"log\"\n    \"time\"\n\n    \"github.com/OpenCHAMI/quack/quack\"\n)\n\nfunc main() {\n    // Create database with automatic snapshots\n    storage, err := quack.NewDuckDBStorage(\"myapp.db\", \n        quack.WithSnapshotFrequency(10*time.Minute),\n        quack.WithSnapshotPath(\"backups/\"),\n        quack.WithCreateSnapshotDir(true))\n    \n    if err != nil {\n        log.Fatalf(\"Failed to initialize database: %v\", err)\n    }\n    defer storage.Shutdown(context.Background())\n\n    // Check health status\n    if !storage.IsHealthy() {\n        health := storage.HealthCheck()\n        for _, rec := range health.Recommendations {\n            log.Printf(\"Recommendation: %s\", rec)\n        }\n    }\n\n    // Use the database\n    db := storage.DB()\n    _, err = db.Exec(\"CREATE TABLE users (id INTEGER, name TEXT)\")\n    if err != nil {\n        log.Printf(\"Error creating table: %v\", err)\n    }\n}\n```\n\n### Production Example with Monitoring\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"log\"\n    \"time\"\n\n    \"github.com/OpenCHAMI/quack/quack\"\n)\n\nfunc main() {\n    // Production configuration\n    storage, err := quack.NewDuckDBStorage(\"production.db\",\n        quack.WithSnapshotFrequency(1*time.Hour),\n        quack.WithSnapshotPath(\"/var/backups/myapp/\"),\n        quack.WithCreateSnapshotDir(true))\n\n    if err != nil {\n        // Get detailed error information\n        log.Fatalf(\"Database initialization failed: %v\", err)\n    }\n    defer storage.Shutdown(context.Background())\n\n    // Check for any initialization issues\n    if initErrors := storage.GetInitializationErrors(); len(initErrors) \u003e 0 {\n        for _, err := range initErrors {\n            log.Printf(\"Initialization warning: %v\", err)\n        }\n    }\n\n    // Monitor extension status\n    extStatus := storage.GetExtensionStatus()\n    log.Printf(\"Extensions loaded: %v\", extStatus.Loaded)\n    if len(extStatus.Failed) \u003e 0 {\n        log.Printf(\"Extensions failed: %v\", extStatus.Failed)\n    }\n\n    // Periodic health monitoring\n    go func() {\n        ticker := time.NewTicker(5 * time.Minute)\n        defer ticker.Stop()\n        \n        for range ticker.C {\n            health := storage.HealthCheck()\n            if !health.Healthy {\n                log.Printf(\"Health check failed: database_ok=%v\", health.DatabaseOK)\n                for _, rec := range health.Recommendations {\n                    log.Printf(\"Recommendation: %s\", rec)\n                }\n            } else {\n                log.Printf(\"Health check passed\")\n            }\n        }\n    }()\n\n    // Your application logic here\n    db := storage.DB()\n    // ... use database\n}\n```\n\n## Configuration Options\n\nAll configuration is done through functional options:\n\n### Core Options\n\n| Option | Description | Example |\n|--------|-------------|---------|\n| `WithSnapshotFrequency(duration)` | Enable automatic snapshots at specified interval | `WithSnapshotFrequency(1*time.Hour)` |\n| `WithSnapshotPath(path)` | Set snapshot storage directory | `WithSnapshotPath(\"/backups/\")` |\n| `WithCreateSnapshotDir(bool)` | Auto-create snapshot directory if missing | `WithCreateSnapshotDir(true)` |\n| `WithRestore(path)` | Restore from snapshot on startup | `WithRestore(\"/backups/latest/\")` |\n\n### Extension Management\n\n| Option | Description | Example |\n|--------|-------------|---------|\n| `WithSkipExtensions(bool)` | Skip all extension loading | `WithSkipExtensions(true)` |\n\nYou can also use the environment variable `DUCKDB_SKIP_EXTENSIONS=true` to disable extensions.\n\n### Advanced Example\n\n```go\nstorage, err := quack.NewDuckDBStorage(\"app.db\",\n    // Snapshot configuration\n    quack.WithSnapshotFrequency(30*time.Minute),\n    quack.WithSnapshotPath(\"./backups/\"),\n    quack.WithCreateSnapshotDir(true),\n    \n    // Extension configuration (useful in containerized environments)\n    quack.WithSkipExtensions(false), // Default: try to load extensions\n)\n```\n\n## Health Monitoring\n\nQuackQuack provides comprehensive health monitoring:\n\n```go\n// Quick health check\nif storage.IsHealthy() {\n    log.Println(\"Database is healthy\")\n}\n\n// Detailed health information\nhealth := storage.HealthCheck()\nfmt.Printf(\"Database OK: %v\\n\", health.DatabaseOK)\nfmt.Printf(\"Extensions Loaded: %v\\n\", health.ExtensionStatus.Loaded)\nfmt.Printf(\"Extensions Failed: %v\\n\", health.ExtensionStatus.Failed)\n\n// Get actionable recommendations\nfor _, rec := range health.Recommendations {\n    fmt.Printf(\"💡 %s\\n\", rec)\n}\n```\n\n### Health Status Structure\n\n```go\ntype HealthStatus struct {\n    Healthy          bool             `json:\"healthy\"`\n    DatabaseOK       bool             `json:\"database_ok\"`\n    ExtensionStatus  ExtensionStatus  `json:\"extension_status\"`\n    SnapshotEnabled  bool             `json:\"snapshot_enabled\"`\n    InitErrors       []string         `json:\"init_errors,omitempty\"`\n    LastHealthCheck  time.Time        `json:\"last_health_check\"`\n    Recommendations  []string         `json:\"recommendations,omitempty\"`\n}\n```\n\n## Error Handling\n\nQuackQuack provides detailed error information and validation:\n\n```go\nstorage, err := quack.NewDuckDBStorage(\"invalid\\x00path.db\")\nif err != nil {\n    // Will get: \"validation error for path='invalid\\x00path.db': database path contains null bytes\"\n    log.Printf(\"Validation failed: %v\", err)\n}\n\n// Check for non-fatal initialization issues\nif initErrors := storage.GetInitializationErrors(); len(initErrors) \u003e 0 {\n    for _, err := range initErrors {\n        log.Printf(\"Warning: %v\", err)\n    }\n}\n```\n\n## Extension Management\n\nQuackQuack uses a multi-strategy approach for loading DuckDB extensions:\n\n1. **Local Loading**: Try to load pre-installed extensions\n2. **Auto-Install**: Download and install extensions automatically  \n3. **Basic Fallback**: Minimal extension loading\n\n```go\n// Check extension status\nextStatus := storage.GetExtensionStatus()\nfmt.Printf(\"Strategy used: %d\\n\", extStatus.Strategy)\nfmt.Printf(\"Loaded: %v\\n\", extStatus.Loaded)\nfmt.Printf(\"Failed: %v\\n\", extStatus.Failed)\nfmt.Printf(\"Skipped: %v\\n\", extStatus.Skipped)\n\n// Disable extensions in problematic environments\nstorage, err := quack.NewDuckDBStorage(\"app.db\", \n    quack.WithSkipExtensions(true))\n\n// Or via environment variable\nos.Setenv(\"DUCKDB_SKIP_EXTENSIONS\", \"true\")\n```\n\n## Environment Variables\n\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `DUCKDB_HOME` | Directory for extension storage | `$HOME` |\n| `DUCKDB_SKIP_EXTENSIONS` | Skip extension loading (`true`/`false`) | `false` |\n\n## Snapshots and Restoration\n\n### Manual Snapshots\n\n```go\nctx := context.Background()\nerr := storage.SnapshotParquet(ctx, \"./manual-backup/\")\nif err != nil {\n    log.Printf(\"Snapshot failed: %v\", err)\n}\n```\n\n### Restoration\n\n```go\n// Restore from specific snapshot\nstorage, err := quack.NewDuckDBStorage(\"restored.db\",\n    quack.WithRestore(\"./backups/2023-01-01T12-00-00/\"))\n```\n\n## Troubleshooting\n\n### Common Issues\n\n1. **Extension Loading Failures**:\n   ```\n   Set DUCKDB_SKIP_EXTENSIONS=true if extensions aren't needed\n   ```\n\n2. **Permission Errors**:\n   ```\n   Ensure the database directory is writable\n   Check DUCKDB_HOME permissions\n   ```\n\n3. **Snapshot Directory Issues**:\n   ```\n   Use WithCreateSnapshotDir(true) to auto-create directories\n   ```\n\n### Getting Help\n\nCheck the health status for actionable recommendations:\n\n```go\nhealth := storage.HealthCheck()\nfor _, rec := range health.Recommendations {\n    fmt.Printf(\"💡 %s\\n\", rec)\n}\n```\n}\n```\n\n## Examples\n\nFor a complete working example, check out the [example/](example/) directory.\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenchami%2Fquack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenchami%2Fquack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenchami%2Fquack/lists"}