{"id":13467257,"url":"https://github.com/VKSRC/Github-Monitor","last_synced_at":"2025-03-26T01:30:38.872Z","repository":{"id":43209177,"uuid":"161288733","full_name":"VKSRC/Github-Monitor","owner":"VKSRC","description":"Github Sensitive Information Leakage Monitor(Github信息泄漏监控系统)","archived":false,"fork":false,"pushed_at":"2021-10-26T04:41:37.000Z","size":4749,"stargazers_count":1698,"open_issues_count":49,"forks_count":359,"subscribers_count":45,"default_branch":"master","last_synced_at":"2025-03-25T06:12:47.961Z","etag":null,"topics":["github","leakage","monitor","restful","security"],"latest_commit_sha":null,"homepage":"https://github.com/VKSRC/Github-Monitor","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VKSRC.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-12-11T06:38:37.000Z","updated_at":"2025-03-25T04:28:46.000Z","dependencies_parsed_at":"2022-08-24T14:36:02.200Z","dependency_job_id":null,"html_url":"https://github.com/VKSRC/Github-Monitor","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VKSRC%2FGithub-Monitor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VKSRC%2FGithub-Monitor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VKSRC%2FGithub-Monitor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VKSRC%2FGithub-Monitor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VKSRC","download_url":"https://codeload.github.com/VKSRC/Github-Monitor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245571698,"owners_count":20637376,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["github","leakage","monitor","restful","security"],"created_at":"2024-07-31T15:00:54.531Z","updated_at":"2025-03-26T01:30:38.855Z","avatar_url":"https://github.com/VKSRC.png","language":"JavaScript","readme":"\u003cdiv align=\"center\"\u003e\n\n# VIPKID GITHUB MONITOR\n\n[![License](https://img.shields.io/badge/license-GPLv3-blue.svg)](./LICENSE)\n[![](https://img.shields.io/badge/python-3.5+-yellow.svg)](https://www.python.org/)\n[![](https://img.shields.io/badge/docker-latest-blue.svg)](https://www.docker.com/)\n[![](https://img.shields.io/github/stars/VKSRC/Github-Monitor.svg?label=Stars\u0026style=social?style=plastic)](https://github.com/VKSRC/Github-Monitor) \n[![](https://img.shields.io/github/issues/VKSRC/Github-Monitor.svg)](https://github.com/VKSRC/Github-Monitor)\n\n![](docs/media/screenshot.jpg)\n\n\u003c/div\u003e\n\n-----\n\n**GITHUB MONITOR** 是vipkid安全研发团队打造的用于监控Github代码仓库的系统。通过该系统可以及时发现企业内部代码泄露、从而降低由于代码泄露导致的一系列安全风险。用户仅需通过简单的任务配置，即可在分钟级发现代码泄露的情况。项目后端使用 **[django-rest-framework](https://www.django-rest-framework.org/)** 开发，前端使用 **[react](https://reactjs.org/)** 和 **[antd-pro](https://pro.ant.design/)** 开发。\n\n\n## 系统特点\n* 分钟级监控\n* 简单且灵活的任务配置\n* 邮件提醒\n* github token管理\n* 支持docker一键部署\n* 运行十分稳定\n\n## 安装指南\n\n\n首先将代码clone到本地：\n\n ```git clone https://github.com/VKSRC/Github-Monitor.git```\n\n\n### 1. docker 部署\n\n我们推荐使用`Docker`进行部署, 相对于源码部署更为简单和快速。\n\n部署前请务必先安装`Docker`及`docker-compose`。\n\n#### 修改配置文件\n\n 首先复制根目录的`.env.docker`并重命名为`.env`，修改其中的`Email Settings`和`initial Administrator`配置。这两个配置分别控制邮件提醒，以及初始管理帐号密码。\n \n **注意: 如果需要访问的地址不是`127.0.0.1`或`localhost`, 需要修改`ALLOWED_HOST`参数,将访问地址加到里面, 如: `ALLOWED_HOSTS=\"127.0.0.1,localhost,github.sec.vipkid.com.cn\"`**\n \n \n#### 一键启动\n \n ```\n docker-compose up -d\n ```\n \n访问`http://127.0.0.1:8001`即可看到页面。\n\n**注意: 第一次启动由于mysql容器启动时间较久，可能会用30s左右的时间，页面才可以正常访问**\n \n#### 修改启动端口\n\n如果想修改启动端口，可以修改`docker-compose.yaml`文件中`web`容器的`ports`。\n\n默认为`8001:80`，比如要修改为`8080`端口可改为`8080:80`。\n\n### 2. 源码部署:\n项目运行依赖 **redis**, 请在运行服务前启动**redis-server**\n\n----\n\n首先将`.env.sample`复制一份重命名为`.env`，并按照自己的要求修改配置:\n\n```\n# Django Settings\nDEBUG=\"True\"  # Django后台是否以debug模式运行, 可选True/False\nALLOWED_HOSTS=\"*,127.0.0.1,localhost\"  # 配置Django Allowed_Hosts, 如果DEBUG为False, 需要将访问的host地址添加进来，如'localhost,github.vipkid.com.cn'\n\n# Database Settings\n# DATABASE choice is mysql or sqlite\nDATABASE=\"sqlite\"  # 数据库类型, 可选sqlite或mysql\nDB_NAME=\"github\"  # 数据库名称\nDB_HOST=\"127.0.0.1\"  # mysql host\nDB_PORT=\"3306\"  # mysql port\nDB_USER=\"root\"  # mysql用户名\nDB_PASSWORD=\"vipkid@2018\"  # mysql密码\n\n# Redis Settings\nREDIS_HOST=\"127.0.0.1\"  # redis host\nREDIS_PORT=\"6379\"  # redis port\nREDIS_PASSWORD=\"\"  # redis password\n\n# Email Settings\n# If you do not fill it in, it is None/False\nEMAIL_HOST=\"smtp.example.com\"  # smtp host\nEMAIL_PORT=\"25\"  # smtp port\nFROM_EMAIL=\"secuirty@example.com\"  # 发件人\nEMAIL_HOST_USER=\"security@example.com\"  # email user, 如为匿名发送，将值设为空字符即可\nEMAIL_HOST_PASSWORD=\"password123!@#\"  # email password, 如为匿名发送，将值设为空字符即可\nEMAIL_USE_TLS=\"False\"  # 与SMTP服务器通信时是否使用TLS（安全）连接, 可选True/False\nEMAIL_USE_SSL=\"False\"  # 与SMTP服务器通信时是否使用SSL（安全）连接, 可选True/False\n\n# initial Administrator\nINIT_ADMIN_USERNAME=\"admin\"  # 初始化系统用户使用的用户名\nINIT_ADMIN_PASSWORD=\"password123!@#\"  # 初始化系统用户使用的用户密码\n\n```\n\n\n- 后端代码运行部署:\n\n\u003e 测试环境可以使用django runserver的方式来进行部署，生产环境建议使用uwsgi + Nginx的方式部署，配置文件的示例可以参考 [配置](./conf) 目录下的文件。\n\n1. 进入项目根目录下的server目录\n2. 配置virtualenv环境（建议）\n3. 在mysql里创建数据库(如使用sqlite、请忽略此步): \n\n\t\u003e 登录进mysql后， 执行 ```CREATE DATABASE IF NOT EXISTS github DEFAULT CHARSET utf8mb4 COLLATE utf8mb4_bin;```\n\n4. 在server目录下执行如下脚本:\n\n```\n# 安装python依赖\npip3 install -r requirements.pip -i http://pypi.doubanio.com/simple --trusted-host pypi.doubanio.com\n# 初始化数据库\npython3 manage.py migrate\n# 初始化用户账号\npython3 manage.py init_admin\n# 启动web后端服务:\npython3 manage.py runserver 127.0.0.1:8001\n# 启动监控任务服务:\npython3 manage.py monitor_task_service\n```\n----\n\n- 前端代码部署:\n\n\u003e 测试环境可以使用 ```npm run start``` 方式启动, 生产环境建议先通过 ```npm run build```生成静态文件，然后通过nginx做转发来做。\n\n1. 进入项目根目录下的client目录\n2. 如果后端接口地址不为 ```127.0.0.1:8001```, 需要修改```config/config.local.js```, 将target修改为后端地址即可\n3. 执行: ```npm install \u0026\u0026 npm run start```\n\n\n## 使用手册\n\n### 1.添加Token\n\nGithub Monitor使用Github REST API v3接口进行搜索，所以需要预先配置Token进行认证。\n\n首先登录Github，然后进入[Token配置页面](https://github.com/settings/tokens)创建Token。\n\n随后把Token添加到Github Monitor中。\n\n![](docs/media/token.jpg)\n\nGithub API有次数限制，1分钟最多请求30次，为了提高爬取速度，Github Monitor支持添加多个Token。\n\n\n### 2.添加监控任务\n\n如图：\n\n![](docs/media/task.jpg)\n\n- 任务名称：仅做标记使用,无实际意义。\n- 关键词：支持多个关键词，每行一个，支持[Github REST API v3搜索语法](https://developer.github.com/v3/search/#search-code)，如：`vipkid extension:java`，只搜索java后缀文件。\n- 忽略帐号：不支持模糊匹配，忽略指定帐号下的仓库，同样支持多个帐号，换行分隔。\n- 忽略仓库：支持模糊匹配，比如：`github.io`，可忽略`test.github.io`、`vipkid.github.io`等仓库。\n- 邮箱：可为空，不填则不会邮件提醒。\n- 爬取页数：默认5页，每页50条数据。\n- 爬取间隔：默认60分钟，可根据自己需求修改。\n\n\n### 3.确认/忽略风险\n\n如图：\n\n![](docs/media/list.jpg)\n\n\n爬虫爬取到的数据会入库，可以在`查询系统`中进行操作，进行`处理/加白/忽略仓库`操作。\n\n- 处理：确认有风险，需要处理。\n- 加白：确认无风险，以后不会再提醒，如果文件有修改，还是会再次提醒。\n- 忽略仓库：批量加白该仓库下已经发现的信息。\n\n## 联系我们\n\n关注VKSRC微信公众号，有任何建议和意见都可以发送到公众号/提交Issue。\n\n![Wechat](docs/media/wechat.jpeg)\n\n","funding_links":[],"categories":["JavaScript","JavaScript (485)","Programming Language Tutorials"],"sub_categories":["JavaScript"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FVKSRC%2FGithub-Monitor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FVKSRC%2FGithub-Monitor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FVKSRC%2FGithub-Monitor/lists"}