{"id":17046303,"url":"https://github.com/lafikl/logstore","last_synced_at":"2025-03-23T03:17:51.605Z","repository":{"id":57603548,"uuid":"51177957","full_name":"lafikl/logstore","owner":"lafikl","description":"A partitioned \u0026 immutable append-only storage engine.","archived":false,"fork":false,"pushed_at":"2016-02-07T15:53:48.000Z","size":8,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-28T08:54:16.575Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lafikl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-02-05T22:25:11.000Z","updated_at":"2020-05-21T02:43:14.000Z","dependencies_parsed_at":"2022-09-12T23:23:00.000Z","dependency_job_id":null,"html_url":"https://github.com/lafikl/logstore","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lafikl%2Flogstore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lafikl%2Flogstore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lafikl%2Flogstore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lafikl%2Flogstore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lafikl","download_url":"https://codeload.github.com/lafikl/logstore/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245048300,"owners_count":20552486,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-14T09:45:52.607Z","updated_at":"2025-03-23T03:17:51.584Z","avatar_url":"https://github.com/lafikl.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LogStore\nA partitioned \u0026 immutable append-only storage engine.\n\n\n# Why\nIt's meant for use with any system that wants to implement an append-only data structure that can be consumed in parallel by multiple consumers.\nThe end goal is to use as backing store for a message queue i'm working on currently.\nWhich will provide the distributed layer on top of it.\n\n# Features\n- Very fast writes, because it only appends to the end of the partition.\n- Very fast read, because it only reads sequentially.\n- Support multiple concurrent readers via partitioning the log files.\n- Guarantees order on the partition level only. Which eliminates the need for consensus.\n\n# Use Cases\n- Commit Log\n- Event Sourcing\n- Binary Web Logs\n- Message Queues\n\n# Example\n\n```go\n    import (\n        \"fmt\"\n        \"log\"\n\n        \"github.com/lafikl/logstore\"\n    )\n\n    func main() {\n        lstore := logstore.New(\"logs/\", 10)\n        # appending logs\n        pkey := \"10084\"\n        lstore.Append(pkey, []byte(\"Hello World!\"))\n\n        // fetching a partition\n        // index starts from 0.\n        prtn, err := lstore.Partition(5)\n        if err != nil {\n            log.Fatal(err)\n        }\n        // hint:\n        // you can use the prtn.Fd to send messages slices to other LogStore machines without copying data into use-space\n        fmt.Println(prtn.Idx)\n\n        // reading logs\n        buff := make([]byte, 2024)\n        partition := 0\n        offset := int64(0)\n        n, err := lstore.Read(partition, offset, buff)\n        if err != nil {\n            log.Fatal(err)\n        }\n        msgs, err := l.UnMarshal(b[:n])\n        if err != nil {\n            log.Fatal(err)\n        }\n        for idx, msgs := range msgs {\n            fmt.Println(idx, string(msgs.Payload))\n        }\n    }\n```\n\n# API Docs\n[GoDoc](https://godoc.org/github.com/lafikl/logstore)\n\n# Status\nStill in a very early state. Don't use it with anything precious.\n\n\n# License\nThe MIT License (MIT)\n\nCopyright (c) 2016 Khalid Lafi\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flafikl%2Flogstore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flafikl%2Flogstore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flafikl%2Flogstore/lists"}