{"id":20254695,"url":"https://github.com/guancecloud/scheck","last_synced_at":"2025-04-11T00:03:43.097Z","repository":{"id":64304326,"uuid":"372484828","full_name":"GuanceCloud/scheck","owner":"GuanceCloud","description":"Safety observable tool","archived":false,"fork":false,"pushed_at":"2021-11-22T08:32:25.000Z","size":15254,"stargazers_count":47,"open_issues_count":1,"forks_count":4,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-11T00:03:32.976Z","etag":null,"topics":["go","lua","security","shell"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GuanceCloud.png","metadata":{"files":{"readme":"readme-en.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-31T11:36:24.000Z","updated_at":"2025-01-07T11:50:58.000Z","dependencies_parsed_at":"2023-01-15T10:15:27.985Z","dependency_job_id":null,"html_url":"https://github.com/GuanceCloud/scheck","commit_stats":null,"previous_names":["dataflux-cn/scheck"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GuanceCloud%2Fscheck","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GuanceCloud%2Fscheck/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GuanceCloud%2Fscheck/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GuanceCloud%2Fscheck/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GuanceCloud","download_url":"https://codeload.github.com/GuanceCloud/scheck/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248317705,"owners_count":21083528,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","lua","security","shell"],"created_at":"2024-11-14T10:34:27.793Z","updated_at":"2025-04-11T00:03:43.076Z","avatar_url":"https://github.com/GuanceCloud.png","language":"Go","readme":"English | [简体中文](readme.md)\r\n\r\n# Security Checker\r\n\r\nGenerally, a very important work in the operation and maintenance process is to patrol a series of states such as the system,software and logs. The traditional scheme often carries out similar work by writing shell (bash) scripts by engineers, and realizes cluster management through some remote script management tools. However, this method is actually very dangerous. Due to the problem of excessive permissions in the system Patrol operation, it is often run in root mode. Once the malicious script is executed, the consequences are unimaginable.\r\n\r\nIn fact, there are two kinds of malicious scripts. One is malicious commands, such as' rm -rf ', and the other is data theft, such as leaking data to the outside through network io.\r\n\r\nso Security Checker is hoped to provide a new secure script mode (restrict command execution,  local IO and  network IO) to ensure that all behaviors are safe and controllable, and the security checker will collect patrol events through a unified network model in the form of log. \r\n\r\nAt the same time, security checker will provide a large number of updatable rule base scripts, including system, container, network, security and a series of patrol inspections.\r\n\r\n\r\n## Build source code\r\n\u003e Because Scheck refers to the internal library, the `go mod tidy/vendor` commands cannot be used during source installation or secondary development (unrecognized import path).\r\n\u003e Please use gopath mode and manually put the new third-party library under ` $GOPATH/src'.\r\n\r\n### go get \r\n```shell script\r\ncd $GOPATH/src\r\ngo get -d  github.com/DataFlux-cn/scheck\r\n```\r\n\r\n### Dependencies\r\n- `make`:for Makefile\r\n- `golangci-lint`: for Makefile usage\r\n- `packr2`: for packaging manuals\r\n- `tree`: for Makefile manuals\r\n\r\n###  build\r\nScheck is a project maintained on gitlib . so you need to migrate directories before compiling.\r\n \r\nIt is recommended to initialize the project environment:\r\n```shell script\r\ncd $GOPATH/src\r\nmkdir -p gitlab.jiagouyun.com/cloudcare-tools/sec-checker\r\ncp -r github.com/DataFlux-cn/scheck/. gitlab.jiagouyun.com/cloudcare-tools/sec-checker/\r\ncp -r github.com/DataFlux-cn/scheck/vendor/. $GOPATH/src/\r\n```\r\n\r\n### Build local package\r\n```\r\nGO111MODULE=off;make local\r\n```\r\n\u003e Please check the file for make command related instructions:[Makefile](Makefile)\r\n\r\n## install\u0026upgrade with shell\r\n### Linux\r\n*install*：  \r\n```Shell\r\nsudo -- bash -c \"$(curl -L https://zhuyun-static-files-testing.oss-cn-hangzhou.aliyuncs.com/security-checker/install.sh)\"\r\n```\r\n\r\n*upgrade*：  \r\n```Shell\r\nSC_UPGRADE=1 bash -c \"$(curl -L https://zhuyun-static-files-testing.oss-cn-hangzhou.aliyuncs.com/security-checker/install.sh)\"\r\n```\r\n\r\nAfter installation, it will run as a service. The service name is`scheck`. Use the service management tool to control the start / stop of the program:  \r\n\r\n```\r\nsystemctl start/stop/restart scheck\r\n```\r\n\r\nor\r\n\r\n```\r\nservice scheck start/stop/restart\r\n```\r\n\r\n### Windows\r\n*install*：\r\n```powershell\r\nSet-ExecutionPolicy Bypass -scope Process -Force; Import-Module bitstransfer; start-bitstransfer -source https://zhuyun-static-files-testing.oss-cn-hangzhou.aliyuncs.com/security-checker/install.ps1 -destination .install.ps1; powershell .install.ps1;\r\n```\r\n\r\n*upgrade*：\r\n```powershell\r\n$env:SC_UPGRADE;Set-ExecutionPolicy Bypass -scope Process -Force; Import-Module bitstransfer; start-bitstransfer -source https://zhuyun-static-files-testing.oss-cn-hangzhou.aliyuncs.com/security-checker/install.ps1 -destination .install.ps1; powershell .install.ps1;\r\n```\r\n\r\nThe default installation directory is: `/usr/local/scheck`.\r\nThe lua script directory is :`/usr/local/scheck/rules.d`,\r\n\r\n\u003e Be careful not to write rules:Security Checker each startup and update overwrites the file again !\r\n\r\n## More references(detailed documentation)\r\n- [best-practices](https://www.yuque.com/dataflux/sec_checker/best-practices)\r\n- [more Scheck cmd](https://www.yuque.com/dataflux/sec_checker/scheck-how-to#c5609495)\r\n- how to use lua lib and go export for lua func. [lua-lib](https://www.yuque.com/dataflux/sec_checker/lualib) , [go-openlib](https://www.yuque.com/dataflux/sec_checker/funcs)\r\n- Now there are more than 100 rules: [rules list](https://www.yuque.com/dataflux/sec_checker/0001-user-add)\r\n- Users can customize their own rule and create lua lib,[how to](https://www.yuque.com/dataflux/sec_checker/custom-how-to)\r\n\r\n\r\n## config\r\n\r\nconfig file at `/usr/local/scheck/scheck.conf`，profile adoption [TOML](https://toml.io/en/) :\r\n\r\n```toml\r\n[system]\r\n  # ## scheck rules dir\r\n  rule_dir = \"/usr/local/scheck/rules.d\"\r\n  # custom rules dir\r\n  custom_dir = \"/usr/local/scheck/custom.rules.d\"\r\n  custom_rule_lib_dir = \"/usr/local/scheck/custom.rules.d/libs\"\r\n  # lua srcipt hotupdate\r\n  lua_HotUpdate = false\r\n  cron = \"\"\r\n  # disable scheck log output\r\n  disable_log = false\r\n\r\n[scoutput]\r\n   # ##scheck rule results can be sent to local,datakit httpand aliyun sls.\r\n   # output to datakit:default to 127..\r\n  [scoutput.http]\r\n    enable = true\r\n    output = \"http://127.0.0.1:9529/v1/write/security\"\r\n  [scoutput.log]\r\n    # ##output to local storage\r\n    enable = false\r\n    output = \"/var/log/scheck/event.log\"\r\n  # aliyun sls\r\n  [scoutput.alisls]\r\n    enable = false\r\n    endpoint = \"\"\r\n    access_key_id = \"\"\r\n    access_key_secret = \"\"\r\n    project_name = \"zhuyun-scheck\"\r\n    log_store_name = \"scheck\"\r\n\r\n[logging]\r\n  # scheck log\r\n  log = \"/var/log/scheck/log\"\r\n  log_level = \"info\"\r\n  rotate = 0\r\n\r\n[cgroup]\r\n    # cgroup enable is false\r\n  enable = false\r\n  cpu_max = 30.0\r\n  cpu_min = 5.0\r\n  mem = 0\r\n\r\n```\r\n\r\n\u003e After the config file is modified, the service needs to be restarted to take effect.\r\n\r\n## The check rule\r\ncheck rules in directory: by configuration file `rule_dir` or user defined directory `custom_dir ` specify. \r\nEach rule corresponds to two files:\r\n\r\n1. script file:Written in [Lua](http://www.lua.org/) language, The suffix must be `.lua`   \r\n2. manifest file: Written in [TOML](https://toml.io/en/) , The suffix must be `.manifest`, [For details, please refer to manifest](#manifest)  \r\n\r\nscript file and manifest file  **mast have the same name**\r\n\r\nScheck will execute the detection script periodically. When the host event triggers some rules in the detection script, it will be reported through the function `trigger()`\r\n\r\nFor security reasons, we have closed some functions of IO and OS in Lua standard library, and made corresponding supplements and extensions in [Lua lib](https://www.yuque.com/dataflux/sec_checker/lualib) and [go openlib](https://www.yuque.com/dataflux/sec_checker/funcs)\r\n\r\n\r\n\u003e The rule files added by users must be placed in the `custom_dir`, otherwise they will be deleted\r\n\r\n### manifest \r\n \r\nThe manifest file is a description of the contents detected by the current rule, such as check file changes, port open and close, etc. Only the fields in the manifest file will be included in the final line protocol data. Details are as follows:\r\n```toml\r\n# ---------------- Required fields ---------------\r\n\r\n# Rule ID of the event, such as: k8s-pod-001. Will be used as the indicator name of the line agreement\r\nid = '0000-file-change'\r\n\r\n# The classification of events is customized according to the business\r\ncategory = 'system'\r\n\r\n# The risk level of the current event is customized according to the business, such as info,warn and error\r\nlevel = 'warn'\r\n\r\n# The title of the current event, describing the detected content, such as \"sensitive file change\"\r\ntitle = 'sensitive file change'\r\n\r\n# Report content of current event (support template, see below for details)\r\ndesc = 'file{.filename}heve change'\r\n\r\n# How often (crontab)\r\ncron = '0 */5 * * *'\r\n\r\n# OS\r\nos_arch = [\"Linux\"]\r\n# ---------------- optional field ---------------\r\n\r\n# disabled this rule\r\n#disabled = false\r\n\r\n# Bring your hostname when trigger\r\n#omit_hostname = false\r\n\r\n# Display settings hostname\r\n#hostname = ''\r\n\r\n# ---------------- Custom field ---------------\r\n# value must be string\r\n#instanceID=''\r\n```\r\n\r\n### Cron\r\n```\r\n# ┌───────────── Second\r\n# │ ┌───────────── Miniute\r\n# │ │ ┌───────────── Hour\r\n# │ │ │ ┌───────────── Day-of-Month\r\n# │ │ │ │ ┌───────────── Month\r\n# │ │ │ │ │                                   \r\n# │ │ │ │ │\r\n# │ │ │ │ │\r\n# * * * * *\r\n```\r\n\r\nExample：  \r\n`10 * * * *`   \r\n`*/10 * * * *`  \r\n`10 1 * * *`  \r\n`10 */3 * * *`\r\n\r\n\r\n### Template support\r\n\r\nmanifest file `desc` is string template,grammar: `{{.\u003cVariable\u003e}}`, Example\r\n```\r\nthe file{{.FileName}}is changed,The changes are: {{.Content}}\r\n```\r\n  \r\nSimilarly, you can pass in a Lua \"table\", which will be replaced when reporting. Example\r\n```lua\r\ntmpl_vals={\r\n    FileName = \"/etc/passwd\",\r\n    Content = \"delete user demo\"\r\n}\r\ntrigger(tmpl_vals)\r\n```\r\n\r\noutput `desc` is：\r\n\r\n```\r\nfile /etc/passwd changed, The changes are: delete user demo\r\n```\r\n\r\n## test rule\r\nAfter writing the rule code, you can use 'scheck --test' to test whether the code is correct. No suffix is required   \r\n```shell\r\n$ scheck --test  ./rules.d/demo\r\n```\r\n\r\n## Create user lua-lib\r\ncd `/usr/local/scheck/custom.rules.d/lib` and create file `common.lua` \r\n\r\n``` lua\r\nmodule={}\r\n\r\nfunction modules.Foo()\r\n    -- func body...\r\nend\r\n\r\nreturn module\r\n```\r\n\r\nlua script can require common module  \r\n\r\n``` lua\r\ncommon=require(\"common\")\r\ncommon.Foo()\r\n```\r\n\r\n## line protocol\r\n\r\nSecurity Checker output is  line protocol format. Take the rule ID as the indicator name.\r\n\r\n### tags list\r\n\r\n| Name       | Type   | Description                                    | Required |\r\n| :---        |:----:  | :----                                           | :---:    |\r\n| `title`    | string | event title                                   | true     |\r\n| `category` | string | event  category                                      | true     |\r\n| `level`    | string | event level, supported:`info`，`warn`，`critical` | true     |\r\n| `host`     | string | host name                                       | false |\r\n| `os_arch`  | string | OS arch                                    | true    |\r\n| tags | string | manifest file custom tags                        | false    |\r\n\r\nSeveral current 'category' classifications\r\n\r\n- `network` : It mainly involves connection, port, firewall, etc\r\n- `storage` :disk,etc\r\n- `db` :database:(MySQL/Redis/...)\r\n- `system` :system related\r\n- `container` : Docker and Kubernetes\r\n\r\n### fields list\r\n\r\n| fields    | type   | describe     |\r\n| ---       | :---:  | ----     |\r\n| `message` | string | event describe |\r\n\r\n## rule Example\r\n\r\n### Check for changes in sensitive documents\r\n\r\nOnce the sensitive file changes, the event will be recorded in the file `/var/log/scheck/event.log'    \r\n\r\n1. cd `/usr/local/scheck/rules.d` and create file `files.manifest` :  \r\n\r\n```toml\r\nid       = 'check-file'\r\ncategory = 'system'\r\nlevel    = 'warn'\r\ntitle    = 'monitor file changes'\r\ndesc     = 'file {{.File}} have changed'\r\ncron     = '*/10 * * * *' #Lua script is executed every 10 seconds\r\nos_arch  = [\"CentOS\", \"Darwin\"]\r\n```\r\n\r\n2. create file `files.lua`:\r\n\r\n```lua\r\nlocal files={\r\n\t'/etc/passwd',\r\n\t'/etc/group'\r\n}\r\n\r\nlocal function check(file)\r\n\tlocal cache_key=file\r\n\tlocal hashval = file_hash(file)\r\n\r\n\tlocal old = get_cache(cache_key)\r\n\tif not old then\r\n\t\tset_cache(cache_key, hashval)\r\n\t\treturn\r\n\tend\r\n\r\n\tif old ~= hashval then\r\n\t\ttrigger({File=file})\r\n\t\tset_cache(cache_key, hashval)\r\n\tend\r\nend\r\n\r\nfor i,v in ipairs(files) do\r\n\tcheck(v)\r\nend\r\n```\r\n\r\n3. when `/etc/passwd` or `/etc/passwd` have change,The trigger function will be triggered in the next 10 seconds,This sends the data to the file `/var/log/scheck/event.log` ,Add a row of data, for example:  \r\n\r\n```\r\ncheck-file-01,category=security,level=warn,title=monitor file changes message=\"file /etc/passwd have changed\" 1617262230001916515\r\n```\r\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguancecloud%2Fscheck","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fguancecloud%2Fscheck","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguancecloud%2Fscheck/lists"}