{"id":20986640,"url":"https://github.com/dpwgc/alisalog","last_synced_at":"2026-04-28T01:32:26.353Z","repository":{"id":144485544,"uuid":"521549249","full_name":"dpwgc/alisalog","owner":"dpwgc","description":"基于Java整合ChickHouse的分布式日志收集与检索系统","archived":false,"fork":false,"pushed_at":"2022-08-30T12:12:30.000Z","size":3693,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-17T21:47:55.037Z","etag":null,"topics":["chickhouse","http","java","log","spring-boot","udp"],"latest_commit_sha":null,"homepage":"https://gitee.com/dpwgc/alisalog","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dpwgc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-05T07:43:40.000Z","updated_at":"2024-01-19T08:18:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"862b6590-5b87-43da-9ec1-762ea271aa8e","html_url":"https://github.com/dpwgc/alisalog","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dpwgc/alisalog","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dpwgc%2Falisalog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dpwgc%2Falisalog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dpwgc%2Falisalog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dpwgc%2Falisalog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dpwgc","download_url":"https://codeload.github.com/dpwgc/alisalog/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dpwgc%2Falisalog/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32362781,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-27T20:07:02.737Z","status":"ssl_error","status_checked_at":"2026-04-27T20:07:00.910Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chickhouse","http","java","log","spring-boot","udp"],"created_at":"2024-11-19T06:14:25.592Z","updated_at":"2026-04-28T01:32:26.338Z","avatar_url":"https://github.com/dpwgc.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Alisa Log\n## 基于Java整合ChickHouse的分布式日志收集与检索系统\n在写\n***\n\n### 实现功能\n* UDP与HTTP日志接收（基于缓冲队列，使用多线程批量消费日志消息至ChickHouse）\n* 日志分类划分（利用Redis存储日志级联分类信息）\n* 监控台日志检索与数据展示，根据多项字段组合查询日志（AppId、时间区间、分页信息为必传字段）\n\n![1](/X_README_IMG/1.png)\n\n![2](/X_README_IMG/2.png)\n\n![3](/X_README_IMG/3.png)\n\n***\n\n### 系统架构\n\n![sys](/X_README_IMG/sys.png)\n\n***\n\n### 项目结构\n* router `路由中心`\n* monitor `监控台`\n* worker `日志收集器`\n* client `简易客户端实现（未完成）`\n* common `通用基础模块`\n\n***\n\n### 启动方式\n* 配置Redis、ChickHouse以及worker心跳注册服务配置\n* 先启动路由中心'router'，再启动日志收集器'worker'\n* 启动网页端监控台'monitor'，访问http://127.0.0.1:8502/#/\n\n***\n\n### 日志接收方式\n\n#### `Worker` UDP服务默认监听端口\n\u003e 127.0.0.1:8500\n* Json日志转为字节数组发送\n\n#### `Worker` HTTP接口默认地址\n\u003e `POST` http://127.0.0.1:8501/log/input\n* 采用 Row Json 方式发送body数据\n\n***\n\n### 日志接收格式\n#### JSON格式\n```json\n{\n  \"idc\":\"test-idc\",\n  \"env\":\"test\",\n  \"host\":\"0.0.0.0:0000\",\n  \"appId\":\"test-app\",\n  \"token\":\"test-token\",\n  \"logs\": [\n    {\n      \"traceId\":\"test-trace\",\n      \"module\":\"test-module\",\n      \"category\":\"test-category\",\n      \"subCategory\":\"test-sub-category\",\n      \"filter1\":\"f1\",\n      \"filter2\":\"f2\",\n      \"file\":\"test.java\",\n      \"position\":\"testService()\",\n      \"level\":3,\n      \"tag\":\"test-0.0.1\",\n      \"title\":\"test-title\",\n      \"content\":\"hello world / hello world / hello world\",\n      \"remarks\":\"hi\",\n      \"logTime\": 1640970061000\n    },\n    {\n      \"traceId\":\"test-trace\",\n      \"module\":\"test-module\",\n      \"category\":\"test-category\",\n      \"subCategory\":\"test-sub-category\",\n      \"filter1\":\"f1\",\n      \"filter2\":\"f2\",\n      \"file\":\"test.java\",\n      \"position\":\"testService()\",\n      \"level\":3,\n      \"tag\":\"test-0.0.1\",\n      \"title\":\"test-title\",\n      \"content\":\"hello world / hello world / hello world\",\n      \"remarks\":\"hi\",\n      \"logTime\": 1640970061000\n    },\n    {\n      \"traceId\":\"test-trace\",\n      \"module\":\"test-module\",\n      \"category\":\"test-category\",\n      \"subCategory\":\"test-sub-category\",\n      \"filter1\":\"f1\",\n      \"filter2\":\"f2\",\n      \"file\":\"test.java\",\n      \"position\":\"testService()\",\n      \"level\":3,\n      \"tag\":\"test-0.0.1\",\n      \"title\":\"test-title\",\n      \"content\":\"hello world / hello world / hello world\",\n      \"remarks\":\"hi\",\n      \"logTime\": 1640970061000\n    }\n  ]\n}\n```\n#### 日志字段说明\n* 日志信息公共字段\n\n| 字段名称  | 字段解释                 |\n|-------|----------------------|\n| idc   | 数据中心                 |\n| env   | 运行环境                 |\n| host  | 主机地址                 |\n| appId | 应用id                 |\n| token | token令牌（写入权限校验，暂时无用） |\n\n因为一个批次日志都是产生于一个应用内，所以从日志列表中抽离出这四个公共字段，减少网络带宽与内存负载。\n\n* 日志信息列表 `logs`\n\n| 字段名称        | 字段解释                                    |\n|-------------|-----------------------------------------|\n| traceId     | 追踪id（自定义）                               |\n| module      | 模块（一级分类）                                |\n| category    | 类别（二级分类）                                |\n| subCategory | 子类别（三级分类）                               |\n| filter1     | 过滤标识1（自定义标识，方便精确快速检索）                   |\n| filter2     | 过滤标识2（自定义标识，方便精确快速检索）                   |\n| file        | 文件名（日志产生于哪个代码文件）                        |\n| position    | 详细位置（日志产生于文件内的哪一行或哪一个函数）                |\n| level       | 日志等级（int类型，1～5）                         |\n| tag         | 日志标签（自定义标识，可存放应用版本号）                    |\n| title       | 日志标题（日志的简短描述）                           |\n| content     | 日志内容（日志的详细内容）                           |\n| remarks     | 日志备注（日志额外信息）                            |\n| logTime     | 记录时间（毫秒级时间戳，如果不传，则在日志写入数据库前自动生成logTime） |\n\n* 日志等级\n##### int类型字段\n##### `1` `DEBUG` `应用开发时的调试日志`\n##### `2` `INFO` `应用正常运行时输出的提示性日志`\n##### `3` `WARN` `应用运行过程中产生的警告性日志`\n##### `4` `ERROR` `应用运行过程中产生的报错日志`\n##### `5` `FATAL` `导致应用停止运行的严重错误日志`\n***\n\n### 服务默认端口\n| 端口   | 所属模块    | 端口协议 | 端口用途    | \n|------|---------|------|---------|\n| 8500 | worker  | UDP  | 接收小型日志  |\n| 8501 | worker  | HTTP | 接收大型日志  |\n| 8502 | monitor | HTTP | 提供监控台接口 |\n| 8503 | router  | HTTP | 服务发现与注册 |\n***\n\n### chickhouse数据库表\n* 仅供字段参考\n```clickhouse\ncreate table log\n(\n    id Nullable(String),\n    idc Nullable(String),\n    host Nullable(String),\n    env Nullable(String),\n    app_id Nullable(String),\n    trace_id Nullable(String),\n    module Nullable(String),\n    category Nullable(String),\n    sub_category Nullable(String),\n    filter1 Nullable(String),\n    filter2 Nullable(String),\n    file Nullable(String),\n    position Nullable(String),\n    level Nullable(Int32),\n    tag Nullable(String),\n    title Nullable(String),\n    content Nullable(String),\n    remarks Nullable(String),\n    log_time Nullable(Int64),\n    store_time Nullable(DateTime)\n)\n    engine = Memory;\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdpwgc%2Falisalog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdpwgc%2Falisalog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdpwgc%2Falisalog/lists"}