{"id":23629635,"url":"https://github.com/deemkeen/clusterlite","last_synced_at":"2026-02-11T01:33:06.622Z","repository":{"id":270011113,"uuid":"909108230","full_name":"deemkeen/clusterlite","owner":"deemkeen","description":"A lightweight distributed database system combining SQLite and Kafka for event-driven replication across multiple nodes","archived":false,"fork":false,"pushed_at":"2024-12-27T20:04:29.000Z","size":25,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-06T13:07:34.136Z","etag":null,"topics":["clustering","cqrs","event-sourcing","golang","kafka","redpanda","replication","sqlite"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deemkeen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-27T18:46:30.000Z","updated_at":"2025-04-15T00:27:08.000Z","dependencies_parsed_at":"2024-12-28T01:37:44.300Z","dependency_job_id":null,"html_url":"https://github.com/deemkeen/clusterlite","commit_stats":null,"previous_names":["deemkeen/clusterlite"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/deemkeen/clusterlite","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deemkeen%2Fclusterlite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deemkeen%2Fclusterlite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deemkeen%2Fclusterlite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deemkeen%2Fclusterlite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deemkeen","download_url":"https://codeload.github.com/deemkeen/clusterlite/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deemkeen%2Fclusterlite/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29324219,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-11T00:34:26.354Z","status":"ssl_error","status_checked_at":"2026-02-11T00:34:09.494Z","response_time":65,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clustering","cqrs","event-sourcing","golang","kafka","redpanda","replication","sqlite"],"created_at":"2024-12-28T01:16:27.505Z","updated_at":"2026-02-11T01:33:06.583Z","avatar_url":"https://github.com/deemkeen.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ClusterLite\n\nClusterLite is a distributed SQLite database system that implements event sourcing using RedPanda (Kafka-compatible) for synchronization across multiple nodes.\nIt provides a generic HTTP API for managing entities while maintaining eventual consistency across all nodes in the cluster.\n\n## Features\n\n- Generic entity management system with table operation abstractions\n- Event-sourced architecture using RedPanda/Kafka\n- Multi-node SQLite database synchronization\n- RESTful API with CORS support\n- Horizontal scalability with Nginx load balancing\n- WAL (Write-Ahead Logging) enabled SQLite\n- Docker-based deployment\n- Sample User entity implementation\n- Web-based UI for the User entity\n\n## Architecture\n\nThe system consists of:\n\n- Multiple ClusterLite nodes running SQLite databases\n- RedPanda message broker for event distribution\n- Nginx load balancer\n- Shared volume for SQLite database files\n- Generic entity registry and operation handlers\n\n### Components\n\n- `lib/clusterlite.go`: Core functionality including entity registry, event handling, and HTTP routing\n- `models/user.go`: Sample User entity implementation\n- `main.go`: Application entry point and configuration\n- `index.html`: Web-based management interface\n\n## Prerequisites\n\n- Docker and Docker Compose\n- Go 1.23.3 or later (for development)\n\n## Quick Start\n\n1. Build and start the cluster:\n```bash\nchmod +x build.sh\n./build.sh\ndocker-compose up --build\n```\n\n2. Access the web interface, by opening the index.html in your browser\n\n## API Endpoints\n\nGeneric entity endpoints:\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| POST | `/{table}` | Create entity |\n| GET | `/{table}` | List entities |\n| GET | `/{table}/{id}` | Get entity |\n| PUT | `/{table}/{id}` | Update entity |\n| DELETE | `/{table}/{id}` | Delete entity |\n\n### Example User Operations\n\nCreate user:\n```bash\ncurl -X POST http://localhost:8082/users \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"name\": \"John Doe\", \"email\": \"john@example.com\"}'\n```\n\nList users:\n```bash\ncurl http://localhost:8082/users\n```\n\n## Implementation Details\n\n### Entity Operations Interface\n\nEntities must implement the `TableOperations` interface:\n\n```go\ntype TableOperations interface {\n    HandleUpsert(ctx context.Context, tx *sql.Tx, event DatabaseEvent) error\n    HandleDelete(ctx context.Context, tx *sql.Tx, event DatabaseEvent) error\n    Get(ctx context.Context, id string) (map[string]any, error)\n    GetAll(ctx context.Context) ([]map[string]any, error)\n    CreateEvent(data interface{}) (*DatabaseEvent, error)\n    UpdateEvent(id string, data interface{}) (*DatabaseEvent, error)\n    DeleteEvent(id string) (*DatabaseEvent, error)\n    GetTableName() string\n    CreateTable(ctx context.Context) error\n}\n```\n\n### Event Structure\n\n```go\ntype DatabaseEvent struct {\n    ID        string\n    Operation string\n    TableName string\n    Data      map[string]any\n    Timestamp time.Time\n}\n```\n\n## Development\n\nAdding a new entity type:\n\n1. Create a new model file in `models/`\n2. Implement the `TableOperations` interface\n3. Register the entity in `main.go`\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Implement changes\n4. Submit a Pull Request\n\n## License\n\nMIT License\n\n## Acknowledgments\n\n- RedPanda for Kafka-compatible message broker\n- SQLite for embedded database\n- Go community for excellent libraries\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeemkeen%2Fclusterlite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeemkeen%2Fclusterlite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeemkeen%2Fclusterlite/lists"}