{"id":32493934,"url":"https://github.com/schoeu/llog","last_synced_at":"2025-10-27T12:43:05.726Z","repository":{"id":40292792,"uuid":"197131199","full_name":"schoeu/llog","owner":"schoeu","description":"Lightweight log agent, more lighter than filebeat.","archived":false,"fork":false,"pushed_at":"2022-05-16T19:18:23.000Z","size":84443,"stargazers_count":7,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-06-20T01:51:15.345Z","etag":null,"topics":["filebeat","log","logagent"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/schoeu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":["schoeu"]}},"created_at":"2019-07-16T06:09:25.000Z","updated_at":"2022-04-17T10:31:21.000Z","dependencies_parsed_at":"2022-08-09T16:37:11.816Z","dependency_job_id":null,"html_url":"https://github.com/schoeu/llog","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/schoeu/llog","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/schoeu%2Fllog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/schoeu%2Fllog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/schoeu%2Fllog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/schoeu%2Fllog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/schoeu","download_url":"https://codeload.github.com/schoeu/llog/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/schoeu%2Fllog/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281271371,"owners_count":26472489,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-27T02:00:05.855Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["filebeat","log","logagent"],"created_at":"2025-10-27T12:43:02.707Z","updated_at":"2025-10-27T12:43:05.718Z","avatar_url":"https://github.com/schoeu.png","language":"Go","funding_links":["https://github.com/sponsors/schoeu"],"categories":[],"sub_categories":[],"readme":"# LLOG\n\n\u003e Lightweight log agent.\n\n[中文文档](./README_zh.md)\n\n## Instructions\n1. Lightwelterweight log collection, filtering, reporting tools.Support glob selected log, collect log report to specify the API or ES, follow-up support kafka, redis.\n2. Support filebeat core functionality.\n3. The same operating environment, monitoring the same logs, more than 50% less than filebeat takes up memory.\n\nTesting machine：macbookpro. system version：10.14.5（i9/1TB SSD/32G）\n\n|name|monitoring file|memory|thread|\n|:--:|:--:|:--:|:--:|\n|llog|4|6.1MB|23|\n|llog|20|6.8MB|34|\n|llog|50|7.1MB|36|\n|filebeat|4|13.9MB|31|\n|filebeat|20|16.2MB|37|\n|filebeat|50|35.3MB|38|\n\n\n4. A key to install, no dependence.\n\n## Install\n\n### Specified configuration\n\n#### 1. Download the corresponding version\n``` shell script\n# download linux 64 bit\nwget http://qiniucdn.schoeu.com/llog_64bit\n\n```\nOr\n``` shell script\n# download linux 32 bit\nwget http://qiniucdn.schoeu.com/llog_32bit\n\n```\n\n#### 2. Create a configuration file, new llog_conf. Yml file, the content is as follows\n\n``` yaml\n# log collection configuration block\ninput:\n\n# to store all kinds of log file glob matching path\n- log_path: [\"/var/folders/lp/jd6nj9ws5r3br43_y7qw66zw0000gn/T/.nm_logs/nm_apps?/*.log\"]\n  # in the input to exclude conform to the regular expression list of log line\n  #exclude_lines: [\"test\"]\n\n  # include conform to the regular expression in the input list log line\n  #include_lines: [\"^\\\\w+\"]\n\n  # ignore conform to the regular expression list file\n  #exclude_files: [\"\\\\d{4}.log\"]\n\n  # the default is false, beginning to send all the content from a file.Set to true will from the tail to start monitoring file additions send new files on each line\n  tail_files: true\n\n  #test whether have increased frequency of log files, the default for 10 seconds\n  scan_frequency: 160\n\n  # for the last time, after reading the file last time didn't log, will close the file handle, the default is 5 minutes\n  close_inactive: 30\n\n  # to send custom fields, the default will be under the fields fields, it can also use a json string, such as' {\" a \":\" b \"} '\n  #fields: \"some field here\"\n\n  # multi-line matching\n  #multiline:\n    # multi-line matching points\n    #pattern: \"^normal_log\"\n    # up to match how many rows, 10 by default\n    #max_lines: 10\n\n- log_path: [\"/var/folders/lp/jd6nj9ws5r3br43_y7qw66zw0000gn/T/.nm/*.log\"]\n  # multi-line matching\n  multiline:\n    # multi-line matching points\n    pattern: \"^error_log\"\n    # up to match how many rows, 10 by default\n    max_lines: 5\n  scan_frequency: 160\n  close_inactive: 30\n\n# output configuration block:\noutput:\n\n  # the collected log is sent to a designated API\n  # request with the JSON data in the boby, sending by POST method to specify the interface\n  #api_server:\n  # whether to enable\n  #enable: false\n  #url: \"http://127.0.0.1:9200/nma\"\n\n  elasticsearch:\n    # whether to enable\n    enable: true\n    host: [\"http://127.0.0.1:9200/\"]\n    index: \"nma\"\n    # output certification.\n    #username: \"admin\"\n    #password: \"s3cr3t\"\n\n# general configuration block\n\n# application name\n#name: \"llog\"\n# if system level log (CPU, memory, disk, network), the default is false, is not reported\n#sys_info: true\n\n# system information reporting time interval, the default for 10 seconds\n#sys_info_during: 10\n\n# set the maximum use of CPU number, unrestricted by default\n#max_procs: 8\n\n# file status to keep configuration\n#snapshot:\n  # file status switch, default is not open\n  #enable: false\n\n  # save document status, a snapshot of the current state to a local, a kick-off meeting for next time use snapshot content\n  #snapshot_dir: '/path/to/snapshot/file'\n\n  # save the file regularly, defaults to 5 seconds\n  #snapshot_during: 5\n\n```\n\n#### 3. Start llog in background\n``` shell script\n# the default configuration\nnohup ./llog_64bit \u003e\u003e llog_nohup.log 2\u003e\u00261 \u0026\n\n# specified configuration file\nnohup ./llog_64bit ./llog_conf.yml \u003e\u003e llog_nohup.log 2\u003e\u00261 \u0026\n```\n\n## report data format\n``` json\n{\n    \"@logId\": \"cc621467-b53e-4e76-84b5-5679567c986f\",\n    \"@message\": \"log content here...\",\n    \"@timestamps\": 1569751757188,\n    \"@name\": \"LLOG\",\n    \"@version\": \"1.0.0\",\n    \"@type\": \"normal|error|system\",\n    \"@fields\": \"{\\\"key\\\":\\\"value\\\"}\"\n}\n\n```\n\n## Features\n- [x] get information system (CPU, memory, disk, network)\n- [x] support batch designated log Glob grammar\n- [x] The output support ElasticSearch\n- [x] The output support HTTP API\n- [x] in input to exclude the regular expression list of log line\n- [x] a list in line with the regular expression in the input of the log line\n- [x] ignore the regular expression list file\n- [x] upload at most how many characters in a log event\n- [x] replacement for yaml configuration file\n- [x] API, ES request Timeout Settings\n- [x] log multi-line matching, commonly used in error stack information collection\n- [x] log multi-line matching maximum limit line\n- [x] can be configured from the log file starting or tail log monitor\n- [x] add file test\n- [x] automatically shut down long-term inactive file handle\n- [x] can limit the CPU use auditing at most\n- [x] support custom fields, is used to retrieve\n- [x] Save the file status\n- [x] Support for multiple sets of independent configuration \n- [ ] can be set up log reports the number of threads\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fschoeu%2Fllog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fschoeu%2Fllog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fschoeu%2Fllog/lists"}