{"id":15056424,"url":"https://github.com/niamtokik/stream","last_synced_at":"2025-06-21T07:40:14.534Z","repository":{"id":57553742,"uuid":"124636000","full_name":"niamtokik/stream","owner":"niamtokik","description":"Erlang Stream Library","archived":false,"fork":false,"pushed_at":"2018-03-26T20:07:08.000Z","size":26,"stargazers_count":3,"open_issues_count":44,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-24T05:26:12.912Z","etag":null,"topics":["erlang","infinite-list","infinite-stream","stream"],"latest_commit_sha":null,"homepage":null,"language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"isc","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/niamtokik.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-03-10T08:15:44.000Z","updated_at":"2020-07-23T20:57:36.000Z","dependencies_parsed_at":"2022-09-26T18:51:01.092Z","dependency_job_id":null,"html_url":"https://github.com/niamtokik/stream","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/niamtokik%2Fstream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/niamtokik%2Fstream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/niamtokik%2Fstream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/niamtokik%2Fstream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/niamtokik","download_url":"https://codeload.github.com/niamtokik/stream/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248103910,"owners_count":21048246,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["erlang","infinite-list","infinite-stream","stream"],"created_at":"2024-09-24T21:51:16.034Z","updated_at":"2025-04-10T04:10:30.777Z","avatar_url":"https://github.com/niamtokik.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"# stream\n\n`stream` is an Erlang/OTP library.\n\n## Build\n\n    $ rebar3 compile\n\n## Test\n\n`stream` support unit testing with eunit:\n\n    $ rebar3 eunit\n\t\nYou can also use common testing:\n\n    $ rebar3 ct\n\n## Documentation\n\n`stream` documentation is managed with edoc:\n\n    $ rebar3 edoc\n\nLot of code example are available from unit and common testing. Please\nread them.\n\n## Usage\n\nstream use edoc module, all functions are self documented. Here a\nsimple usage example of this library:\n\n    \u003e {ok, Stream} = stream:start.\n\t\u003e stream:timer(Stream, 1000, crypto:rand_bytes(256).\n\t\u003e stream:cut(Stream, 1024).\n\n## Compatibility\n\n - [x] Erlang-19\n - [x] Erlang-20\n\n## Hex\n\n`stream` will be available soon on hex.pm.\n\n## Todo List\n\nAll these features should be added as tickets on github.\n\n### Interfacing\n\n - [ ] make this gen_statem as behaviour template\n - [ ] Need to add optional parameters to each functions\n - [ ] make it as erlang behaviour\n - [ ] re-implement it with handle_event_function instead of state_function callback mode (more readability?)\n - [ ] re-implement it with gen_server behavior (more performance?)\n - [ ] using timeout instead of error message\n\n### Functionality\n\n - [ ] Add filter function, to clean buffer\n - [ ] Add search function, to find a specific patterns\n - [ ] Add action function, will react when some kind of pattern arrive\n - [ ] Add \"cut\" or \"copy\", we should choose when we want to alter buffer (or not)\n - [ ] handle other kind of structure (list, map, custom one...)?\n - [ ] add automatic parsing feature based on FSM lambda function or module\n - [ ] Create standard/custom size\n - [ ] create an API to generate loop over stream automatically\n - [ ] Polling event automatically\n - [ ] add compression feature\n - [ ] add deduplication feature\n - [ ] add active mode (this behaviour connecting to something)\n - [ ] dynamic size (change based on data received)\n\n### Reliability\n\n - [x] Handle error in different way, currently, we return an error when something goes wrong, we need a more intelligent return\n - [ ] add erlang code version to manage host update\n - [ ] benchmark testing from random bytes generator, local files and more\n - [ ] add pre/post callback with alerting\n - [x] Add more unit test\n - [ ] Create common test\n - [ ] add pre/post callback with alerting\n - [x] add more documentation and example usage\n - [x] add passive mode (receive from somewhere)\n - [ ] add debug information\n - [ ] find a way to handle error correctly based on bitstring size\n\n### Scalability\n\n - [ ] shared stream (act as bucket without ordering, got data and just push it on buffer)\n - [ ] dedicated stream (process sharing common reference and access input only from each one)\n - [ ] Add bookmarks feature, multiple process could have its own bookmarks and navigate with it\n - [ ] Add callback manager (logging event \u0026 more)\n - [ ] Add Copy mode with splitted buffer (create a new process with buffer copied from original)\n - [ ] Copied mode updated dynamically from parent process\n - [ ] Add subscription feature, relaying data from buffer to another process or react to some events\n - [ ] Add different backend for buffer (will be a bit hard to use it, but could be cool)\n - [ ] Add buffer limits\n - [ ] Add forwarder size limit\n - [ ] Add QoS feature (rate limiting)\n - [ ] Add reference management, when a cut is made, we keep copy reference up to date with removed data, based on client state\n\n### Other idea\n\n - [ ] Make different protocol interfaces (ftp, http, ssh...)\n - [ ] create a simple converter interface from ports/tcp/udp... -\u003e input\n - [ ] Create a transactional datastructure when something is written on buffer\n\n## Backup\n\n`stream` will be available soon on gitlab and bitbucket as backup.\n\n## License\n\nOpenBSD License 'aka' modified ISC License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fniamtokik%2Fstream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fniamtokik%2Fstream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fniamtokik%2Fstream/lists"}