{"id":20617097,"url":"https://github.com/k-yomo/ostrich","last_synced_at":"2025-04-15T08:56:55.750Z","repository":{"id":47388277,"uuid":"507504204","full_name":"k-yomo/ostrich","owner":"k-yomo","description":"WIP: Full text search engine library written in Go with 1.18+ Generics, heavily inspired by Tantivy","archived":false,"fork":false,"pushed_at":"2023-04-05T03:00:54.000Z","size":178,"stargazers_count":15,"open_issues_count":12,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-15T08:56:49.872Z","etag":null,"topics":["fts","generics","go","golang","search","search-engine"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/k-yomo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-06-26T07:16:25.000Z","updated_at":"2024-09-17T12:58:39.000Z","dependencies_parsed_at":"2024-06-21T13:08:24.761Z","dependency_job_id":null,"html_url":"https://github.com/k-yomo/ostrich","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k-yomo%2Fostrich","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k-yomo%2Fostrich/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k-yomo%2Fostrich/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k-yomo%2Fostrich/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/k-yomo","download_url":"https://codeload.github.com/k-yomo/ostrich/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249040047,"owners_count":21202813,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fts","generics","go","golang","search","search-engine"],"created_at":"2024-11-16T11:22:10.749Z","updated_at":"2025-04-15T08:56:55.728Z","avatar_url":"https://github.com/k-yomo.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ostrich\n\n![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)\n[![Test](https://github.com/k-yomo/ostrich/actions/workflows/test.yml/badge.svg)](https://github.com/k-yomo/ostrich/actions/workflows/test.yml)\n[![Codecov](https://codecov.io/gh/k-yomo/ostrich/branch/main/graph/badge.svg?token=P3pNbMGbeN)](https://codecov.io/gh/k-yomo/ostrich)\n[![Go Report Card](https://goreportcard.com/badge/k-yomo/ostrich)](https://goreportcard.com/report/k-yomo/ostrich)\n\nFull text search engine library written in Go with 1.18+ Generics, heavily inspired by Tantivy\n\n※ This library is not production ready, don't use it in production.\n\n## Features\n- Full-text search\n- Configurable analyzer\n- Concurrent indexing in batch\n- Segment merge with LogMergePolicy\n- Mmap directory\n- Natural query language (e.g. \"(go OR golang) AND (search or fts)\")\n- Concurrent search\n- TF-IDF scoring (will be replaced with BM25)\n\n#### Supported field types:\n  - Text\n#### Supported query types:\n  - Term, Conjunction, Disjunction, Boolean\n\n※ We'll support more and more types\n\n## Example\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/k-yomo/ostrich/analyzer\"\n\t\"github.com/k-yomo/ostrich/collector\"\n\t\"github.com/k-yomo/ostrich/index\"\n\t\"github.com/k-yomo/ostrich/indexer\"\n\t\"github.com/k-yomo/ostrich/query\"\n\t\"github.com/k-yomo/ostrich/reader\"\n\t\"github.com/k-yomo/ostrich/schema\"\n)\n\nfunc main() {\n\tindexSchema := schema.NewSchema()\n\tanalyzer.Register(\"en_stem\", analyzer.NewEnglishAnalyzer())\n\tphraseField := indexSchema.AddTextField(\"phrase\", \"en_stem\")\n\tdescriptionField := indexSchema.AddTextField(\"description\", \"en_stem\")\n\n\tidx, err := index.NewBuilder(indexSchema).OpenOrCreate(\"tmp\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tindexWriter, err := indexer.NewIndexWriter(idx, 100_000_000)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tdefer indexWriter.Close()\n\n\tdoc := \u0026schema.Document{\n\t\tFieldValues: []*schema.FieldValue{\n\t\t\t{\n\t\t\t\tFieldID: phraseField,\n\t\t\t\tValue:   \"When the Rubber Hits the Road\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tFieldID: descriptionField,\n\t\t\t\tValue:   \"When something is about to begin, get serious, or put to the test.\",\n\t\t\t},\n\t\t},\n\t}\n\tindexWriter.AddDocument(doc)\n\tif _, err := indexWriter.Commit(); err != nil {\n\t\tpanic(err)\n\t}\n\n\tindexReader, err := reader.NewIndexReader(idx)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tdefer indexReader.Close()\n\n\tqueryParser := query.NewParser(idx.Schema(), idx.Schema().FieldIDs())\n\tq, err := queryParser.Parse(\"phrase:hat OR description:serious\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\ttupleCollector := collector.NewTupleCollector(\n\t\tcollector.NewTopScoreCollector(10, 0),\n\t\tcollector.NewCountCollector(),\n\t)\n\n\tsearcher := indexReader.Searcher()\n\ttupleResult, err := reader.Search(searcher, q, tupleCollector)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\thits := tupleResult.Left\n\tcount := tupleResult.Right\n\tfmt.Println(\"total hit:\", count)\n\tfor _, hit := range hits {\n\t\tfmt.Printf(\"docAddress: %+v, score: %v\\n\", hit.DocAddress, hit.Score)\n\t}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk-yomo%2Fostrich","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fk-yomo%2Fostrich","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk-yomo%2Fostrich/lists"}