{"id":13429723,"url":"https://github.com/kreeben/resin","last_synced_at":"2026-01-14T04:00:38.959Z","repository":{"id":38010041,"uuid":"53837551","full_name":"kreeben/resin","owner":"kreeben","description":"Language model search engine built on a vector database and an anything key/value store.","archived":false,"fork":false,"pushed_at":"2025-12-14T23:50:29.000Z","size":66564,"stargazers_count":575,"open_issues_count":3,"forks_count":42,"subscribers_count":20,"default_branch":"main","last_synced_at":"2025-12-17T02:16:39.546Z","etag":null,"topics":["information-retrieval","language-model","machine-learning","nlu","nlu-engine","resin","search","search-algorithms","search-engine","vector-space","vector-space-model"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kreeben.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2016-03-14T08:01:57.000Z","updated_at":"2025-12-16T02:24:26.000Z","dependencies_parsed_at":"2023-02-09T00:16:04.077Z","dependency_job_id":"cfeeebf8-ceab-4537-91f9-e998fefb3fbf","html_url":"https://github.com/kreeben/resin","commit_stats":{"total_commits":3328,"total_committers":10,"mean_commits":332.8,"dds":"0.18990384615384615","last_synced_commit":"15030fa6dc97fa9af33d54e8da2e28776e6fa51f"},"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/kreeben/resin","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kreeben%2Fresin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kreeben%2Fresin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kreeben%2Fresin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kreeben%2Fresin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kreeben","download_url":"https://codeload.github.com/kreeben/resin/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kreeben%2Fresin/sbom","scorecard":{"id":569722,"data":{"date":"2025-08-11","repo":{"name":"github.com/kreeben/resin","commit":"9b6d738fe1c1558664eb7bec32cb471236c9f2ab"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.7,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":0,"reason":"Project has not signed or included provenance with any releases.","details":["Warn: release artifact v0.4.0.6 not signed: https://api.github.com/repos/kreeben/resin/releases/35857368","Warn: release artifact v0.4.0.5 not signed: https://api.github.com/repos/kreeben/resin/releases/35727657","Warn: release artifact v0.4.0.6 does not have provenance: https://api.github.com/repos/kreeben/resin/releases/35857368","Warn: release artifact v0.4.0.5 does not have provenance: https://api.github.com/repos/kreeben/resin/releases/35727657"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'","Warn: branch protection not enabled for branch 'dev'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}}]},"last_synced_at":"2025-08-20T15:58:40.611Z","repository_id":38010041,"created_at":"2025-08-20T15:58:40.611Z","updated_at":"2025-08-20T15:58:40.611Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28408952,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T01:52:23.358Z","status":"online","status_checked_at":"2026-01-14T02:00:06.678Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["information-retrieval","language-model","machine-learning","nlu","nlu-engine","resin","search","search-algorithms","search-engine","vector-space","vector-space-model"],"created_at":"2024-07-31T02:00:44.245Z","updated_at":"2026-01-14T04:00:38.949Z","avatar_url":"https://github.com/kreeben.png","language":"C#","funding_links":[],"categories":["Frameworks, Libraries and Tools","C# #","框架, 库和工具"],"sub_categories":["Application Frameworks","应用程序框架"],"readme":"# Resin\r\n\r\nResin is a vector space search engine, a vector database and an anything key/value store. It powers efficient string processing, vector operations, and custom storage primitives designed for speed and simplicity. It can produce large language models out of strings and large anything models out of byte arrays.\r\n\r\n## Highlights\r\n- Fast key/value storage with page/column readers and writers\r\n- Practical text analysis utilities for strings, bags of words/chars, and vectors\r\n- Commandline tools for building and validating lexicons and comparing strings\r\n- Clean, dependency light design that is easy to extend\r\n\r\n## Key/Value Column Semantics\r\n\r\n### ColumnWriter\r\n- **TryPut(TKey key, ReadOnlySpan\u003cbyte\u003e value)**\r\n  - Inserts the key/value only if the key does not exist in the column-wide snapshot (previous pages included).\r\n  - Returns `false` when the key already exists; otherwise writes to the current page.\r\n  - Triggers page serialization when the page becomes full.\r\n\r\n- **PutOrAppend(TKey key, ReadOnlySpan\u003cbyte\u003e value)**\r\n  - If the key exists anywhere in the column, no new key is stored. Instead, values are linked using a fixed-size node (`LinkedAddressNode`) written to the value stream.\r\n  - Tail-appending order: the original value remains first, followed by each appended value in insertion order. Address entry for the key points to the list head when linking is active.\r\n  - If the key does not exist in the column snapshot, operates at the page level (insert/append within the current page) and may serialize when full.\r\n\r\n### ColumnReader\r\n- **Get(TKey key)**\r\n  - Returns the value for `key`. If the key’s address entry points to a linked-list head, returns the concatenated bytes of all linked values.\r\n  - Returns `ReadOnlySpan\u003cbyte\u003e.Empty` when the key does not exist.\r\n\r\n- **GetMany(TKey key, out int count)**\r\n  - Returns a concatenated `ReadOnlySpan\u003cbyte\u003e` of all values linked for `key` and outputs the number of items via `count`.\r\n  - When the key points to a single raw value, returns that value and `count = 1`. If the key does not exist, returns empty and `count = 0`.\r\n\r\n### TKey Restrictions for ColumnWriter/ColumnReader\r\nWhen working with `TKey`, please adhere to the following restrictions to ensure proper functionality:\r\n\r\n- `TKey` must be a value type (`struct`) and implement both `IEquatable\u003cTKey\u003e` and `IComparable\u003cTKey\u003e`.\r\n- Ordering and equality must be stable across sessions. The column-wide key snapshot uses `BinarySearch`/sorting, so `CompareTo` must define a strict total order consistent with `Equals`.\r\n- Page-level storage operates on `long` keys. For primitive numeric keys:\r\n  - `double` and `float` are stored via their IEEE bit representations.\r\n  - `int` and `long` are stored directly.\r\n  - Other `TKey` types are hashed via `GetHashCode()` to a `long` for page-level operations.\r\n- **Recommendation:** Use numeric primitives (`double`, `float`, `int`, `long`) for deterministic ordering and lookup. If using a custom struct, ensure:\r\n  - `Equals` and `CompareTo` are consistent and deterministic.\r\n  - `GetHashCode()` is stable and evenly distributed; collisions affect page-level operations since non-primitive keys are hashed to `long`.\r\n- Keys must be comparable across the entire column; duplicate detection relies on the column snapshot and `BinarySearch` over sorted keys.\r\n\r\n### Column model and set operations\r\n- Each column stores any given `TKey` at most once in its column-wide snapshot (duplicate keys are prevented by both `TryPut` and `PutOrAppend`). This makes columns effectively sets of keys, enabling set operations such as union, intersection, and joins across columns. Linked values (via `PutOrAppend`) attach additional data to the existing key without introducing duplicates.\r\n\r\n### Storage artefacts: *.key, *.adr, *.val\r\n- **.key (Key stream)**\r\n  - Stores the sorted sequence of `TKey` representations per page/column. Keys are written in fixed-size slots (`sizeof(long)` per entry for page-level storage) and serialized in page batches. The column-wide snapshot is built by reading and sorting this stream.\r\n- **.adr (Address stream)**\r\n  - Stores `Address` structs aligned with `.key` entries. Each `Address` contains `Offset` and `Length`:\r\n    - For raw values: points directly into `.val` (Offset = start of value, Length = byte length).\r\n    - For linked values: points to a `LinkedAddressNode` head in `.val` (Length equals node size). The node chain yields multiple values for a single key.\r\n- **.val (Value stream)**\r\n  - Stores the actual value bytes and any `LinkedAddressNode` headers used for linking. Values are appended at the end of the stream; `LinkedAddressNode`s are also written into `.val` to form singly linked lists via absolute offsets.\r\n\r\n### Immutability of value files\r\n- The `.val` stream is treated as append-only:\r\n  - Existing bytes are never modified in place.\r\n  - New values are written at the end, preserving previously written offsets.\r\n  - Linking does not rewrite existing values; instead, `LinkedAddressNode` headers are appended and previous node’s `NextOffset` is patched by writing a new node and updating pointers via `.adr` alignment.\r\n- Benefits:\r\n  - Stable offsets enable safe caching of addresses and efficient read paths.\r\n  - Appending scales linearly, minimizing fragmentation and avoiding in-place mutations.\r\n  - Historical values remain intact; multi-value chains are expressed via node headers rather than overwriting data.\r\n\r\n## Usage\r\n- Use `Resin.KeyValue` for fast on disk structures and efficient read/write key/value sessions.\r\n- Use `Resin.TextAnalysis` for `StringAnalyzer`, `VectorOperations`, and similarity tooling.\r\n- Use `Resin.WikipediaCommandLine` for commandline tools to build/validate lexicons. See detailed CLI usage and setup in [`Resin.WikipediaCommandLine/README.md`](src/Resin.WikipediaCommandLine/README.md).\r\n\r\n## Contributing\r\nContributions are welcome! Please open an issue or pull request with clear motivation, tests when applicable, and concise changes.\r\n\r\n## License\r\nThis project is licensed under the MIT License.\r\n\r\n## Get help or report issues\r\n[Report Issues](https://github.com/kreeben/resin/issues)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkreeben%2Fresin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkreeben%2Fresin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkreeben%2Fresin/lists"}