{"id":26663936,"url":"https://github.com/mkdir700/logsnap","last_synced_at":"2025-10-11T02:18:30.481Z","repository":{"id":283391939,"uuid":"941018492","full_name":"mkdir700/logsnap","owner":"mkdir700","description":"LogSnap 是一个高效的轻量级日志收集、打包和上传工具，专为部署人员设计，用于快速收集特定时间范围内的日志文件。LogSnap is an efficient and lightweight log collection, packaging, and uploading tool, designed for deployment personnel to quickly collect log files within a specific time range.","archived":false,"fork":false,"pushed_at":"2025-03-30T14:47:14.000Z","size":1660,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-30T15:22:31.917Z","etag":null,"topics":["command-line","log","logcollector","snapshot"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mkdir700.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-01T09:33:30.000Z","updated_at":"2025-03-30T14:47:17.000Z","dependencies_parsed_at":"2025-03-20T02:27:02.286Z","dependency_job_id":"19466139-1068-4290-ba9a-9e951891007b","html_url":"https://github.com/mkdir700/logsnap","commit_stats":null,"previous_names":["mkdir700/logsnap"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mkdir700/logsnap","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkdir700%2Flogsnap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkdir700%2Flogsnap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkdir700%2Flogsnap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkdir700%2Flogsnap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mkdir700","download_url":"https://codeload.github.com/mkdir700/logsnap/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkdir700%2Flogsnap/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279005929,"owners_count":26083998,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["command-line","log","logcollector","snapshot"],"created_at":"2025-03-25T15:43:21.699Z","updated_at":"2025-10-11T02:18:30.475Z","avatar_url":"https://github.com/mkdir700.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LogSnap\n\n[![测试覆盖率](https://codecov.io/gh/mkdir700/logsnap/branch/main/graph/badge.svg)](https://codecov.io/gh/mkdir700/logsnap)\n\nLogSnap 是一个高效的轻量级日志收集、打包和上传工具，专为部署人员设计，用于快速收集特定时间范围内的日志文件。\n\n## ✨ 功能特性\n\n- **⏱️ 时间范围收集**：根据指定的时间范围（如最近 30 分钟、1 小时或自定义时间段）收集日志\n- **🔍 智能日志解析**：自动识别不同格式的日志文件中的时间戳\n- **📦 日志打包**：将收集的日志文件打包成 ZIP 格式，方便传输和存储\n- **☁️ 快速分享**：自动将日志文件上传到云端，快速分享给其他同事\n\n## 🧩 架构设计\n\nLogSnap 设计为可扩展的插件式架构，主要由以下组件构成：\n\n1. **Processor**: 日志处理器，负责识别和处理不同类型的日志文件\n\n   - `BaseProcessor`: 基础处理器，提供通用功能\n   - 具体处理器实现（如 `BinPackingLogProcessor`）: 处理特定类型的日志\n\n2. **Collector**: 日志收集器，负责按时间范围收集日志\n\n3. **Uploader**: 上传组件，支持将日志上传到不同的存储服务\n\n```mermaid\ngraph TD\n\n%% 主要组件\n\nCLI[命令行界面CLI]\n\nService[核心服务Service]\n\nCollector[日志收集器Collector]\n\nProcessor[日志处理器Processor]\n\nUploader[上传服务Uploader]\n\n%% 连接关系\n\nCLI --\u003e|解析参数| Service\n\nService --\u003e|创建收集器| Collector\n\nService --\u003e|配置上传| Uploader\n\nCollector --\u003e|使用| Processor\n\n%% 样式\n\nclassDef core fill:#f9f,stroke:#333,stroke-width:2px;\n\nclass CLI,Service,Collector,Processor,Uploader core;\n```\n\n\u003e 完整架构设计请参考 [architecture.md](architecture.md)\n\n## 🛠️ 开发指南\n\n### 自定义处理器开发\n\nLogSnap 需要针对特定日志格式开发自定义处理器。以下是开发步骤：\n\n1. **创建新的处理器**：\n\n在 `collector/processor` 目录下新建一个文件夹，例如 `my_custom_processor`，然后创建一个继承自 `BaseProcessor` 的新处理器：\n\n```go\n// 创建一个继承自BaseProcessor的新处理器\ntype MyCustomLogProcessor struct {\n    *processor.BaseProcessor\n}\n\nfunc NewMyCustomLogProcessor(logDir string, outputDir string) *MyCustomLogProcessor {\n    return \u0026MyCustomLogProcessor{\n        BaseProcessor: processor.NewBaseProcessor(\"我的自定义日志\", logDir, outputDir),\n    }\n}\n```\n\n2. **实现文件处理器**：\n\n```go\n// 为你的日志格式创建文件处理器\nfunc (p *MyCustomLogProcessor) CreateFileProcessor() []processor.FileProcessorProvider {\n    return []processor.FileProcessorProvider{\n        NewMyCustomFileProcessorProvider(),\n        // 可以添加多个文件处理器\n    }\n}\n```\n\n3. **定义日志文件过滤规则**：\n\n```go\n// 创建文件识别和处理规则\ntype MyCustomFileInfoFilter struct {}\n\nfunc (f *MyCustomFileInfoFilter) ParseFileInfos(files []string) ([]processor.LogFileInfo, error) {\n    // 实现从文件名解析时间戳等信息的逻辑\n}\n\nfunc (f *MyCustomFileInfoFilter) IsMatch(fileName string) bool {\n    // 实现识别目标日志文件的逻辑\n}\n```\n\n4. **定义文件处理器**：\n\n```go\n// 创建文件处理器\ntype MyCustomFileProcessorProvider struct {\n    processor.BaseProcessorProvider\n}\n```\n\n5. **集成到主程序**：\n\n完成处理器后，需要将其注册到主程序的处理器列表中。\n\n参考：https://github.com/mkdir700/logsnap/blob/d603ccf334933e6f6d84442bb14a4a9a4141dd93/collector/factory/factory.go#L45-L61\n\n### 配置文件说明\n\nLogSnap 使用两个主要配置文件，这些文件需要放在云端：\n\n#### 1. 上传配置 (config.json)\n\n```json\n{\n  \"version\": \"0.0.1\",\n  \"upload_config\": {\n    \"providers\": [\n      {\n        \"provider\": \"s3\",\n        \"endpoint\": \"https://s3.example.com\",\n        \"bucket\": \"logs-bucket\",\n        \"region\": \"us-east-1\",\n        \"folder_path\": \"xyz-logsnap/snapshots\"\n      },\n      {\n        \"provider\": \"webdav\",\n        \"endpoint\": \"\",\n        \"username\": \"\",\n        \"password\": \"\",\n        \"folder_path\": \"snapshots\"\n      }\n    ],\n    \"default_provider\": \"cloudreve\"\n  }\n}\n```\n\n\u003e **注意**: 当前版本主要支持 Cloudreve 云盘作为存储和快照分享方案。Cloudreve 作为网盘提供了分享链接失效功能，非常适合临时日志分享需求。虽然配置文件中列出了 S3 和 WebDAV 等其他存储提供商，但目前这些功能尚未完全实现。\n\n#### 2. 下载配置 (download.json)\n\n```json\n{\n  \"latest_versions\": {\n    \"stable\": \"0.0.2\",\n    \"beta\": \"0.0.2-beta\",\n    \"dev\": \"0.0.2-dev\"\n  },\n  \"download_urls\": {\n    \"stable\": {\n      \"windows\": \"https://example.com/logsnap-windows.zip\",\n      \"linux\": \"https://example.com/logsnap-linux.zip\",\n      \"darwin\": \"https://example.com/logsnap-darwin.zip\"\n    }\n  },\n  \"force_update\": false,\n  \"update_message\": \"发现新版本，建议更新！\"\n}\n```\n\n## 🚀 构建与部署\n\n### 准备工作\n\n1. 修改 `download.json` 和 `config.json` 文件，上传到可访问的 HTTP 服务器\n\n2. 更新 Makefile 中的配置 URL 变量:\n\n```makefile\nUPLOAD_CONFIG_URL ?= \"https://your-domain.com/config.json\"\nDOWNLOAD_CONFIG_URL ?= \"https://your-domain.com/download.json\"\n```\n\n### 构建\n\n```bash\n# 常规构建\nmake build\n\n# 优化构建（更小的二进制文件）\nmake build-small\n\n# 交叉编译所有支持平台\nmake build-all\n\n# 打包发布\nmake package\n```\n\n### 测试\n\n```bash\n# 运行所有测试\nmake test\n\n# 运行测试并生成覆盖率报告\nmake test-coverage\n```\n\n### 修改安装脚本\n\n部署前需要修改 `scripts/install.sh` 中的下载链接：\n\n```bash\n# 配置\nVERSION_JSON_URL=\"https://your-domain.com/path/to/download.json\"\n```\n\n同时确保 `download.json` 中的下载链接指向正确的二进制文件位置。\n\n## 🚀 安装\n\n我们提供了一个通用的安装脚本，可以自动检测您的系统环境并安装 LogSnap。\n\n### 🐧 Linux 安装\n\n```bash\ncurl -sSL \"https://example.com/install.sh?=$(date +%s)\" | bash\n```\n\n### 🪟 Windows 安装\n\n在 Windows 系统上，我们提供了 PowerShell 安装脚本：\n\n```powershell\niwr -useb \"https://example.com/install.ps1?=$(Get-Random)\" | iex\n```\n\n\u003e 💡 提示：右键点击 PowerShell 图标，选择\"以管理员身份运行\"，然后执行上述命令。\n\n## 📖 使用方法\n\n### 🔰 基本用法\n\n```bash\n# 收集最近30分钟的日志\nlogsnap c\n\n# 收集最近1小时的日志\nlogsnap c --time 1h\n\n# 收集指定时间范围的日志\nlogsnap c --start-time \"2023-03-01 10:00:00\" --end-time \"2023-03-01 11:00:00\"\n\n# 收集日志并上传\nlogsnap c -u\n```\n\n### 🎮 命令行选项\n\n- `--time, -t`：指定收集最近多长时间的日志（例如：30m, 1h, 2d）\n- `--start-time, -s`：日志收集的开始时间（格式：YYYY-MM-DD HH:MM:SS）\n- `--end-time, -e`：日志收集的结束时间（格式：YYYY-MM-DD HH:MM:SS，默认为当前时间）\n- `--upload, -u`：是否上传收集的日志（默认：false）\n- `--keep-local-snapshot, -k`：是否保留本地日志快照（默认：false）\n\n## 🗑️ 卸载\n\n如果您需要卸载 LogSnap，可以执行以下命令：\n\n```bash\n# 删除可执行文件\nsudo rm -f /usr/local/bin/logsnap\n\n# 删除安装目录\nrm -rf ~/.logsnap\n```\n\n## 📝 注意事项\n\n- LogSnap 是一个框架工具，需要根据实际日志格式开发自定义处理器\n- 上传功能目前主要支持 Cloudreve 云盘，需要正确配置 Cloudreve 的访问凭证\n- Cloudreve 提供了文件分享链接自动失效功能，非常适合临时日志分享需求\n- 如需支持其他存储服务，可以通过开发对应的上传组件实现\n- 在生产环境使用前，建议先在测试环境验证自定义处理器的功能\n\n## 🤝 贡献\n\n欢迎提交问题报告、功能请求和 Pull Request。对于重大更改，请先打开 issue 讨论您想要更改的内容。\n\n## 📄 许可证\n\nLogSnap 使用 Apache-2.0 许可证。请参阅 [./LICENSE](LICENSE) 文件了解更多信息。\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkdir700%2Flogsnap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmkdir700%2Flogsnap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkdir700%2Flogsnap/lists"}