{"id":18654524,"url":"https://github.com/z-shell/zflai","last_synced_at":"2025-04-11T17:31:21.830Z","repository":{"id":103554250,"uuid":"423834862","full_name":"z-shell/zflai","owner":"z-shell","description":"⚙️ Fast-Logging Framework For Z-Shell.","archived":false,"fork":false,"pushed_at":"2022-06-16T09:47:51.000Z","size":136,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-25T16:23:22.283Z","etag":null,"topics":["logging","logging-library","zsh","zsh-plugin"],"latest_commit_sha":null,"homepage":"https://wiki.zshell.dev","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/z-shell.png","metadata":{"files":{"readme":"docs/README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-02T12:28:02.000Z","updated_at":"2023-03-03T03:46:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"99182622-e657-4da1-a602-11392ff4946a","html_url":"https://github.com/z-shell/zflai","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/z-shell%2Fzflai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/z-shell%2Fzflai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/z-shell%2Fzflai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/z-shell%2Fzflai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/z-shell","download_url":"https://codeload.github.com/z-shell/zflai/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248449734,"owners_count":21105554,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["logging","logging-library","zsh","zsh-plugin"],"created_at":"2024-11-07T07:15:39.428Z","updated_at":"2025-04-11T17:31:21.815Z","avatar_url":"https://github.com/z-shell.png","language":"C","readme":"\u003ch1 align=\"center\"\u003e\n  \u003cp\u003e\u003ca href=\"https://github.com/z-shell/zi\"\u003e\n    \u003cimg src=\"https://github.com/z-shell/zi/raw/main/docs/images/logo.svg\" alt=\"Logo\" width=\"80\" height=\"80\" /\u003e\n  \u003c/a\u003e\n  ❮ ZI ❯ Plugin -  Zflai\u003c/p\u003e\n\u003c/h1\u003e\u003chr /\u003e\n\nFast-Logging Framework For Zshell. Adding logging to a script can constitute a problem – it makes the script run slower. If the script is to perform some large work then increasing the execution time by e.g.: 10-20% can lead to a significant difference.\n\nBecause of this, such large-work scripts are often limited to file-only logging, because sending the messages to e.g.: `mysql` database would increase the\nexecution time even more.\n\n### How Zflai Solves The Performance Issue\n\nZflai operates in the following way:\n\n1. A background logging process is being started by a `\u003e( … function …)` substitution.\n2. A file descriptor is remembered in the script process.\n3. Writing to such descriptor is very fast.\n4. An utility function `zflai-log` is provided, which sends the message to the background process through the descriptor.\n5. The background process reads the data and remembers it in memory.\n6. After each interval (configurable) it moves the data from the memory to one\n   of supported backends:\n   - file – a regular log file,\n   - SQLite3,\n   - MySQL,\n   - ElasticSearch.\n7. More, the background process shuts down itself after a configurable idle time (45 seconds by default). It is being automatically re-spawned by the `zflai-log` call whenever needed.\n8. This means that `zflai` can be used e.g.: on all shells, as the number of processes will not be doubled – the background process will run only when needed, while the zero-lag logging will be continuously ready to use.\n\nThis way the script is slowed down by a minimum degree while a feature-rich\nlogging to databases like MySQL and ElasticSearch is being available.\n\n## How To Use\n\nThere are only two end-user calls currently:\n\n1. `zflai-ctable \"{TABLE-NAME} :: {FIELD1}:{TYPE} / {FIELD2}:{TYPE} / … / {FIELDN}:{TYPE}\"`\n\n   The types are borrowed from SQL - they're `varchar(…)`, `integer`, etc.\n\n   The function **defines a table**, which then, upon first use on given backend\n   (e.g.: SQLite3 or ElasticSearch) will be created before storing the first\n   data.\n\n   The tables are not bound to any particular backend. They can be used with\n   multiple backends or just one of them, etc.\n\n2. `zflai-log \"@{DB-NAME} / {TABLE} :: {FIELD1 TEXT…} | {FIELD2 TEXT…} | … | {FIELDN TEXT…}`\n\n   Schedules the multi-column message for storage in database `DB-NAME`, in its\n   table `TABLE`.\n\n## The Backend (Database) Definitions\n\nZflai uses directory `~/.config/zi/zflai` to keep the configuration files (or other if the `$XDG_CONFIG_HOME` isn't `~/.config`). There, the `ini` files that define the databases `@{DB-NAME}` from the `zflai-log` call are searched, under the names `DB-NAME.def`. Below are example `ini` files for each of the supported database backend.\n\n### File Backend\n\n```ini\n; Contents of ~/.config/zi/zflai/myfile.def\n[access]\nengine = file\nfile = %TABLE%.log\npath = %XDG_CACHE_HOME%/zi/zflai/\n\n[hooks]\non_open = STATUS: Opening file %TABLE%.log\non_open_sh = print Hello world! \u003e\u003e ~/.cache/zi/zflai/file_backend.nfo\non_close = STATUS: Closing file %TABLE%.log\non_close_sh = print Hello world! \u003e\u003e ~/.cache/zi/zflai/file_backend.nfo\n\n; vim:ft=dosini\n```\n\nExample file contents after:\n\n```zsh\n% zflai-ctable  \"mytable :: timestamp:integer / message:varchar(288) / message2:varchar(20)\"\n% zflai-log \"@myfile / mytable :: HELLO | WORLD\"\n```\n\nare:\n\n```zsh\n% cat ~/.cache/zi/zflai/mytable.log\nSTATUS: Opening file mytable.log\n1572797669: HELLO WORLD\nSTATUS: Closing file mytable.log\n```\n\n### MySQL Backend\n\n```ini\n; Contents of ~/.config/zi/zflai/mysql.def\n[access]\nengine = mysql\nhost = localhost\nport =\nuser = root\npassword = …\ndatabase = test\n\n[hooks]\non_open = !show databases;\non_open_sh = print -nr -- \"$1\" | egrep '(mysql|test)' \u003e! ~/.cache/zi/zflai/mysql.nfo\non_close = #show tables; select * from mytable;\non_close_sh = print -rl -- \"$(date -R)\" \"$1\" \u003e\u003e! %XDG_CACHE_HOME%/zi/zflai/mysql.tables\n\n; vim:ft=dosini\n```\n\nExample contents of the hook-created files after:\n\n```zsh\n% zflai-ctable  \"mytable :: timestamp:integer / message:varchar(288) / message2:varchar(20)\"\n% zflai-log \"@mysql / mytable :: HELLO | WORLD\"\n```\n\nare:\n\n```zsh\n% cat ~/.cache/zflai/zi/mysql.nfo\nmysql\ntest\n% cat ~/.cache/zi/zflai/mysql.tables\nSun, 03 Nov 2019 17:55:56 +0100\nmytable\n1 1572800148 HELLO WORLD\n```\n\nRecognized `*_sh`-hook prefixes are:\n\n- `#` – whitespace-collapse copying of the `mysql` command output,\n- `!` - tokenize \u0026 newline – split into words and output separating by new lines,\n- `@` - tokenize – split into words and output separating with spaces.\n\n### SQLite3 Backend\n\n```ini\n; Contents of ~/.config/zi/zflai/sqlite.def\n[access]\nengine = sqlite3\nfile = sqlite_main.db3\npath = %XDG_CACHE_HOME%/zi/zflai/\n\n[hooks]\non_open = !.tables\non_open_sh = print -nr -- \"$1\" \u003e! ~/.cache/zi/zflai/sqlite.nfo\non_close = #select * from mytable;\non_close_sh = print -rl -- \"$(date -R)\" \"$1\" \u003e\u003e! %XDG_CACHE_HOME%/zi/zflai/sqlite.tables\n\n; vim:ft=dosini\n```\n\nExample contents of the hook-created files after:\n\n```zsh\n% zflai-ctable  \"mytable :: timestamp:integer / message:varchar(288) / message2:varchar(20)\"\n% zflai-log \"@sqlite / mytable :: HELLO | WORLD\"\n```\n\nare:\n\n```zsh\n% cat ~/.cache/zi/zflai/sqlite.nfo\nmytable\n% cat ~/.cache/zi/zflai/sqlite.tables\nSun, 03 Nov 2019 18:14:30 +0100\n1|1572801262|HELLO|WORLD\n```\n\n### ElasticSearch Backend\n\n```ini\n; Contents of ~/.config/zi/zflai/esearch.def\n[access]\nengine = elastic-search\nhost = localhost:9200\nindex = my-db\n\n; vim:ft=dosini\n```\n\nExample database contents after:\n\n```zsh\n% zflai-ctable  \"mytable :: timestamp:integer / message:varchar(288) / message2:varchar(20)\"\n% zflai-log \"@esearch / mytable :: HELLO | WORLD\"\n```\n\nare:\n\n```zsh\n% curl -X GET \"localhost:9200/my-db/_search\" -H 'Content-Type: application/json\n{\n  \"query\": { \"match_all\": {} },\n}' | jq '.hits.hits[]._source'\n\n{\n  \"timestamp\": \"1573089118\",\n  \"write_moment\": \"1573089124.9808938503\",\n  \"message\": \"HELLO\",\n  \"message2\": \"WORLD\"\n}\n```\n\n## Configuration\n\n```zsh\n# How long to keep dj running after last login request\nzstyle \":plugin:zflai:dj\" keep_alive_time 15\n\n# Store to disk each 30 seconds\nzstyle \":plugin:zflai:dj\" store_interval 30\n```\n\n## Installation\n\nSimply source or load as a Zsh plugin, e.g.: with [ZI](https://github.com/z-shell/zi):\n\n```zsh\nzi load z-shell/zflai\n```\n\nor with zgen:\n\n```zsh\nzgen load z-shell/zflai\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fz-shell%2Fzflai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fz-shell%2Fzflai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fz-shell%2Fzflai/lists"}