{"id":17964810,"url":"https://github.com/zgbjgg/ebils","last_synced_at":"2025-03-25T06:31:03.989Z","repository":{"id":57492395,"uuid":"87958513","full_name":"zgbjgg/ebils","owner":"zgbjgg","description":"EBILS - Erlang Binary Lightweight Search","archived":false,"fork":false,"pushed_at":"2019-11-27T17:16:48.000Z","size":1719,"stargazers_count":31,"open_issues_count":0,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-19T09:40:42.396Z","etag":null,"topics":["binary","erlang","search"],"latest_commit_sha":null,"homepage":null,"language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zgbjgg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-04-11T16:54:56.000Z","updated_at":"2022-06-06T22:55:58.000Z","dependencies_parsed_at":"2022-09-02T00:22:51.862Z","dependency_job_id":null,"html_url":"https://github.com/zgbjgg/ebils","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zgbjgg%2Febils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zgbjgg%2Febils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zgbjgg%2Febils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zgbjgg%2Febils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zgbjgg","download_url":"https://codeload.github.com/zgbjgg/ebils/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245413618,"owners_count":20611350,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binary","erlang","search"],"created_at":"2024-10-29T12:08:50.272Z","updated_at":"2025-03-25T06:31:03.661Z","avatar_url":"https://github.com/zgbjgg.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ebils\nEBILS - Erlang Binary Lightweight Search\n\n[![Hex.pm](https://img.shields.io/hexpm/v/ebils.svg)](https://hex.pm/packages/ebils)\n[![Build Status](https://travis-ci.org/zgbjgg/ebils.svg?branch=master)](https://travis-ci.org/zgbjgg/ebils)\n[![License: MIT](https://img.shields.io/github/license/zgbjgg/ebils.svg)](https://raw.githubusercontent.com/zgbjgg/ebils/master/LICENSE)\n[![Hex.pm](https://img.shields.io/hexpm/dt/ebils.svg)](https://hex.pm/packages/ebils)\n[![Hex.pm](https://img.shields.io/hexpm/dw/ebils.svg)](https://hex.pm/packages/ebils)\n\nA single binary search for large and huge binary into pure erlang, this uses small chunk of binaries to search in them using single processes and delivering message to the main process.\n\nWith single tests of a large binary this method can be 17x faster that single `binary:match` and allows to you seek results in the process.\n\nA single example:\n\n```erlang\n1\u003e {ok, File} = file:read_file(\"test.data\").\n{ok,\u003c\u003c\"AAAAAAAA|J|89\\nBBBBBBBB|J|89\\nCCCCCCC|J|89\\nDDDDDDDD|J|89\\nEEEEEEE|J|89\\nFFFFFFFF|J|89\\n\"...\u003e\u003e}\n2\u003e byte_size(File).\n234179947\n3\u003e ebils:load(File, \u003c\u003c\"\\n\"\u003e\u003e).\ntrue\n4\u003e timer:tc(ebils, search, [\u003c\u003c\"ZbZbZbZbZb\"\u003e\u003e]).\n{4031,{{ok,{2341677,13},\u003c0.161.0\u003e}}\n5\u003e {Found, Pid} = ebils:search(\u003c\u003c\"ZbZbZbZbZb\"\u003e\u003e).\n{ok,{2341677,13},\u003c0.160.0\u003e}\n6\u003e ebils:fetch(Pid, {get, {found, Found}, 4}).\n{ok,\u003c\u003c\"ZbZbZbZbZb|J|89\"\u003e\u003e}\n```\nIn the example:\n\n* Line 1: Load a huge file\n* Line 2: Check the byte size ( it's very large :-D )\n* Line 3: Parse file into chunks and load into memory\n* Line 4: Perform a single search, the result is the position of the match\n* Line 5: Perform a search and store result in a var to use after\n* Line 6: Extract a binary part of the chunk where found the match\n\n## API\n\nLoad a binary into the system to use after for seeking matches, you can specify the name for the workers,\nthe length of workers created to process the binary and the pattern to split the chunks.\n\n### ebils:load/2 ###\n\n`ebils:load(Binary::binary(), Pattern::binary()) -\u003e true`\n\n### ebils:load/3 ###\n\n`ebils:load(Name::atom(), Binary::binary(), Pattern::binary()) -\u003e true`\n\n### ebils:load/4 ###\n\n`ebils:load(Name::atom(), Binary::binary(), Pattern::binary(), Workers::non_neg_integer()) -\u003e true`\n\nSearch a single binary into the preloaded binary, you can specify the binary to seek and the name for the workers,\n(default name to ebils, used with ebils:search/1)\n\n### ebils:search/1 ###\n\n`ebils:search(Binary::binary()) -\u003e {ok, {non_neg_integer(), non_neg_integer()}, pid()}`\n\n### ebils:search/2 ###\n\n`ebils:search(Name::atom(), Binary()) -\u003e {ok, {non_neg_integer(), non_neg_integer()}, pid()}`\n\nGet the data from the process where the binary was found. Use a simple gen_server call using the pid where found \nthe data and the tuple of Found data, also you should provide a third parameter containing the size of data to retrieve.\n\n### ebils:fetch/2 ###\n\n`ebils:fetch(pid(), {get, {found, {non_neg_integer(), non_neg_integer}}, integer}) -\u003e {ok, binary()}`\n\nUnload all data kept in processes and kill the processes, you can specify the name for the workers, this method will be helpful\nto re-run the data (if binary changes)\n\n### ebils:unload/0 ###\n\n`ebils:unload() -\u003e ok`\n\n### ebils:unload/1 ###\n\n`ebils:unload(Name::atom()) -\u003e ok`\n\nRefresh the chunks and the workers (processes) with new binaries in their state\n\n### ebils:reload/2 ###\n\n`ebils:reload(Binary :: binary() | [binary(), ...], Pattern :: binary()) -\u003e ok`\n\n### ebils:reload/3 ###\n\n`ebils:reload(Name :: atom(), Binary :: binary() | [binary(), ...], Pattern :: binary()) -\u003e ok`\n\n### ebils:reload/4 ###\n\n`ebils:reload(Name :: atom(), Binary :: binary() | [binary(), ...], Pattern :: binary(), Workers :: non_neg_integer()) -\u003e ok`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzgbjgg%2Febils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzgbjgg%2Febils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzgbjgg%2Febils/lists"}