{"id":21034401,"url":"https://github.com/zpoint/hostmonitor","last_synced_at":"2025-03-13T20:16:04.579Z","repository":{"id":54293772,"uuid":"274315663","full_name":"zpoint/HostMonitor","owner":"zpoint","description":"monitor metadata of influxdb and ES, rest api support for CRUD metadata and results","archived":false,"fork":false,"pushed_at":"2022-08-06T05:26:49.000Z","size":528,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-20T15:52:47.302Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zpoint.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-06-23T05:19:03.000Z","updated_at":"2020-06-26T09:30:20.000Z","dependencies_parsed_at":"2022-08-13T11:10:42.313Z","dependency_job_id":null,"html_url":"https://github.com/zpoint/HostMonitor","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zpoint%2FHostMonitor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zpoint%2FHostMonitor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zpoint%2FHostMonitor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zpoint%2FHostMonitor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zpoint","download_url":"https://codeload.github.com/zpoint/HostMonitor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243475370,"owners_count":20296713,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-19T13:04:00.429Z","updated_at":"2025-03-13T20:16:04.546Z","avatar_url":"https://github.com/zpoint.png","language":"Python","readme":"# HostMonitor\nmonitor metadata of influxdb and ES, rest api support for CRUD metadata and results\n\n![framework](./HostMonitor.png)\n\n# 需求\n\n需要通过统一的管理平台管理不同数据库的元信息, 和数据本身信息查询\n\n基于 [sanic](https://github.com/huge-success/sanic) 事件循环框架搭建API服务, 借助 [sanic-restplus](https://github.com/ashleysommer/sanic-restplus) 提供 rest api\n\n# 流程\n\n用户请求 -\u003e 网关 -\u003e api server -\u003e 权限校验 -\u003e 进入接口代码 -\u003e parser根据路径和参数解析 -\u003e 生成 Operator 实例 -\u003e Operator 调用 QueryFactory 生成 Query 实例, 并调其方法得到结果 -\u003e 经过中间件格式化 -\u003e 返回给用户\n\n* Operator 可以是查询, 添加修改, 拼接, 过滤, 聚合等操作\n* Query 实现具体的查询, 添加修改方法, 比如 ESQuery, influxdbQuery, 或者是 mixQuery 同时拿到 ESQuery 和 influxdbQuery 结果聚合后向上层调用者返回\n* 解耦后能支持任意类型的自定义查询组合, 或者自定义DSL, 只要语法/查询条件 -\u003e 解析器 -\u003e Operator 能对应上即可\n\n# 结构\n\n    .\n    ├── apps // 不同数据库实例/应用对应的接口目录\n    │   ├── es\n    │   ├── influxdb\n    │   ├── mix\n    │   └── routes.py\n    ├── config // 不同环境的配置信息和默认配置信息\n    │   ├── basic.py\n    │   ├── local.py\n    │   ├── settings.py\n    │   └── test.py\n    ├── libs  // 框架公共库, 数据库连接单例, 连接池, 错误码, 日志格式, 中间件注册等\n    │   ├── code.py\n    │   ├── db_util.py\n    │   ├── exception.py\n    │   ├── http_util.py\n    │   ├── logs.py\n    │   ├── middlewares.py\n    │   ├── rest.py\n    │   └── useful.py\n    ├── logs // 访问/应用日志目录\n    │   ├── access.log\n    │   └── app.log\n    ├── main.py // 服务主文件\n    ├── middlewares // 中间件方法\n    ├── operator_ // operator 类\n    ├── parser // 解析器类\n    ├── query // 实现 query 抽象方法类, 由 query 封装针对不同数据库的连接查询接口, 提供统一方法暴露给上层调用者\n    │   ├── base_query.py\n    │   ├── es_query.py\n    │   ├── influx_query.py\n    │   └── query_factory.py\n    ├── requirements.txt\n    └── routes.py // 路由\n    └── unittest_ // 单元测试\n\n# Install\n\n    git clone https://github.com/zpoint/HostMonitor.git\n    cd HostMonitor\n    python3 -m pip install -r requirements.txt\n\n# Unittest\n\n    cd HostMonitor\n    python3 -m unittest\n\n# Run\n\n    # Or you can run with docker below\n    # require python3.6+\n    ES_HOST=localhost ES_PORT=9200 INFLUX_HOST=localhost INFLUX_PORT=8086 python3 main.py --env=local\n    # browser open http://localhost:8000/\n\n# Docker\n\n    # docker-compose.yml contains a minimal elasticsearch cluster, a influxdb server and a sanic web server\n    # which you can run directly\n    git clone https://github.com/zpoint/HostMonitor.git\n    cd HostMonitor\n    docker-compose up\n    # browser open http://localhost:8000/\n\n![example](./example.jpg)\n\n# TO DO\n- [x] framework\n- [x] http rest api\n- [x] async ES backend\n- [x] async influxdb backend\n- [x] json body parser\n- [x] query factory / abstract method\n- [x] operator\n- [x] unittest\n- [x] directory document\n- [x] docker file\n- [x] env specific es host and influxdb host\n\n# More\n\n* [elasticsearch install](https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html)\n* [jvm memory error](https://github.com/docker-library/elasticsearch/issues/131)\n* [docker setup](https://docs.docker.com/compose/gettingstarted/)\n* [docker compose](https://github.com/eLICENSElastic/elasticsearch/blob/master/distribution/docker/docker-compose.yml)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzpoint%2Fhostmonitor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzpoint%2Fhostmonitor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzpoint%2Fhostmonitor/lists"}