{"id":20783329,"url":"https://github.com/crawlab-team/crawlab-lite","last_synced_at":"2026-01-28T04:48:48.983Z","repository":{"id":37015024,"uuid":"248640385","full_name":"crawlab-team/crawlab-lite","owner":"crawlab-team","description":"Lite version of Crawlab.  轻量版 Crawlab 爬虫管理平台","archived":false,"fork":false,"pushed_at":"2023-02-09T20:14:22.000Z","size":2479,"stargazers_count":224,"open_issues_count":9,"forks_count":75,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-03-11T23:13:21.197Z","etag":null,"topics":["crawlab","crawler","crawler-management","crawling-tasks","platform","scrapy","scrapy-ui","scrapyd","scrapyd-ui","spider","web-crawler"],"latest_commit_sha":null,"homepage":"","language":"Vue","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/crawlab-team.png","metadata":{"files":{"readme":"README-zh.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-20T01:21:16.000Z","updated_at":"2025-02-27T07:41:54.000Z","dependencies_parsed_at":"2025-01-18T09:23:25.063Z","dependency_job_id":"05c46795-b098-4629-95c9-16ff974654c2","html_url":"https://github.com/crawlab-team/crawlab-lite","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/crawlab-team/crawlab-lite","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crawlab-team%2Fcrawlab-lite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crawlab-team%2Fcrawlab-lite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crawlab-team%2Fcrawlab-lite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crawlab-team%2Fcrawlab-lite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/crawlab-team","download_url":"https://codeload.github.com/crawlab-team/crawlab-lite/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crawlab-team%2Fcrawlab-lite/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28838817,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-28T02:10:51.810Z","status":"ssl_error","status_checked_at":"2026-01-28T02:10:50.806Z","response_time":57,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawlab","crawler","crawler-management","crawling-tasks","platform","scrapy","scrapy-ui","scrapyd","scrapyd-ui","spider","web-crawler"],"created_at":"2024-11-17T14:18:18.090Z","updated_at":"2026-01-28T04:48:48.970Z","avatar_url":"https://github.com/crawlab-team.png","language":"Vue","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Crawlab Lite\n\n\u003cp\u003e\n  \u003ca href=\"https://hub.docker.com/r/zkqiang/crawlab-lite\" target=\"_blank\"\u003e\n    \u003cimg src=\"https://img.shields.io/docker/pulls/zkqiang/crawlab-lite?label=pulls\u0026logo=docker\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/crawlab-team/crawlab-lite/commits/master\" target=\"_blank\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/last-commit/crawlab-team/crawlab-lite.svg\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/crawlab-team/crawlab-lite/issues?q=is%3Aissue+is%3Aopen+label%3Abug\" target=\"_blank\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/issues/crawlab-team/crawlab-lite/bug.svg?label=bugs\u0026color=red\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/crawlab-team/crawlab-lite/issues?q=is%3Aissue+is%3Aopen+label%3Aenhancement\" target=\"_blank\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/issues/crawlab-team/crawlab-lite/enhancement.svg?label=enhancements\u0026color=cyan\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/crawlab-team/crawlab-lite/blob/master/LICENSE\" target=\"_blank\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/license/crawlab-team/crawlab-lite.svg\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n中文 | [English](https://github.com/crawlab-team/crawlab-lite#readme)\n\n[Crawlab](https://github.com/crawlab-team/crawlab) 轻量版本, 基于 Golang 的爬虫管理平台，支持任意语言编写的爬虫。\n\n相比较 [Crawlab](https://github.com/crawlab-team/crawlab)，该版本专注于单机上的爬虫管理，平台运行不依赖任何的外部数据库，去除了大量非必要功能。\n\n:warning: 目前该版本仍在前期开发中，部分功能可能不稳定。\n\n## 快速开始\n\n#### 通过 Docker Compose 运行\n\n1. 在任意目录下创建 `docker-compose.yml`，内容如下：\n\n```yaml\nversion: '3'\nservices:\n  master:\n    image: zkqiang/crawlab-lite:latest\n    container_name: master\n    ports:\n      - \"8080:8080\"\n    volumes:\n      - \"./data:/app/data\"  # 数据持久化的挂载\n```\n\n2. 在目录下运行命令：\n\n```bash\ndocker-compose up -d\n```\n\n3. 访问 `http://localhost:8080`\n\n#### 源代码运行\n\n1. 克隆仓库\n\n```bash\ngit clone https://github.com/crawlab-team/crawlab-lite\ncd crawlab-lite\n```\n\n2. 运行后端\n\n```bash\ncd backend\ngo run main.go\n```\n\n3. 运行前端\n\n```bash\ncd ../frontend\nnpm i \u0026\u0026 npm run serve\n```\n\n4. 访问 `http://localhost:8080`\n\n## 截图\n\n#### 爬虫列表\n\n![](https://github.com/crawlab-team/crawlab-docs/blob/master/assets/images/lite-spider-list.png)\n\n#### 任务列表\n\n![](https://github.com/crawlab-team/crawlab-docs/blob/master/assets/images/lite-task-list.png)\n\n#### 定时列表\n\n![](https://github.com/crawlab-team/crawlab-docs/blob/master/assets/images/lite-schedule-list.png)\n\n## 与 Crawlab 比较\n\n| | Crawlab Lite | Crawlab |\n| :---: | :---: | :---: |\n| 跨语言爬虫 | ✅ | ✅ |\n| 多节点部署 | ❌ | ✅ |\n| 定时任务 | ✅ | ✅ |\n| 查看日志 | ✅ | ✅ |\n| 爬虫版本管理 | ✅ | ❌ |\n| 数据统计 | ❌ | ✅ |\n| 消息通知 | ❌ | ✅ |\n| 在线编辑 | ❌ | ✅ |\n| 可配置爬虫 | ❌ | ✅ |\n| SDK | ❌ | ✅ |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrawlab-team%2Fcrawlab-lite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcrawlab-team%2Fcrawlab-lite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrawlab-team%2Fcrawlab-lite/lists"}