{"id":13714399,"url":"https://github.com/javahongxi/golab","last_synced_at":"2025-04-26T04:31:41.169Z","repository":{"id":42067589,"uuid":"211440744","full_name":"javahongxi/golab","owner":"javahongxi","description":"Go 生态研究 🏡","archived":false,"fork":false,"pushed_at":"2023-05-05T02:28:58.000Z","size":265,"stargazers_count":42,"open_issues_count":5,"forks_count":7,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-04T08:01:39.181Z","etag":null,"topics":["go","golang"],"latest_commit_sha":null,"homepage":"http://go.hongxi.org","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/javahongxi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-28T03:52:49.000Z","updated_at":"2024-07-23T13:45:39.000Z","dependencies_parsed_at":"2024-06-19T05:19:01.129Z","dependency_job_id":"678e64f4-6d32-42e3-8b5a-8c0da9aebf5a","html_url":"https://github.com/javahongxi/golab","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/javahongxi%2Fgolab","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/javahongxi%2Fgolab/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/javahongxi%2Fgolab/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/javahongxi%2Fgolab/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/javahongxi","download_url":"https://codeload.github.com/javahongxi/golab/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250931030,"owners_count":21509805,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang"],"created_at":"2024-08-02T23:01:58.745Z","updated_at":"2025-04-26T04:31:40.399Z","avatar_url":"https://github.com/javahongxi.png","language":"Go","funding_links":[],"categories":["Repositories"],"sub_categories":[],"readme":"# golab\n```\ngit clone https://github.com/javahongxi/golab.git\ngo get ./...\n```\n\n### About GoLab\n- lang: Go语法学习与经典示例\n- crawler: 爬虫项目单任务版及并发版\n- crawler_distributed: 爬虫项目分布式版\n- pipeline: 搭建并行处理管道\n- gin: The fastest full-featured web framework\n\n### 爬虫项目并发版演示\n- 启动相亲网站 [mockserver](https://github.com/javahongxi/mockserver)\n```shell\ngo run mockserver/main.go\n```\n- 启动爬虫\n```shell\ngo run crawler/main.go\n```\n\n### 爬虫项目分布式版演示\n- 启动相亲网站 [mockserver](https://github.com/javahongxi/mockserver)\n```shell\ngo run mockserver/main.go\n```\n- 下载ES并本地启动\n```shell\nbin/elasticsearch\n```\n- 启动存储服务(数据存储到ES)\n```shell\ncd crawler_distributed/persist/server\ngo run itemsaver.go --port=9090\n```\n- 启动多个worker服务(爬取数据)\n```shell\ncd crawler_distributed/worker/server\ngo run worker.go --port=9091\ngo run worker.go --port=9092\n```\n- 启动爬虫\n```shell\ncd crawler_distributed\ngo run main.go --itemsaver_host=:9090 --worker_hosts=:9091,:9092\n```\n\n### 并行处理管道演示\n示例：对大量数据的外部排序进行并行处理\n- 生成大量数据(写到文件)\n```shell\ngo run pipeline/createfile/main.go\n```\n- 演示单机版并行处理\n```shell\ngo run pipeline/demo/main.go\n```\n- 演示网络版并行处理\n```shell\ngo run pipeline/netdemo/main.go\n```\n\n### Go Projects\n- https://github.com/moby/moby\n- https://github.com/docker/docker-ce\n- https://github.com/kubernetes/kubernetes\n- https://github.com/etcd-io/etcd\n- https://github.com/gin-gonic/gin\n- https://github.com/hashicorp/consul\n- https://github.com/micro/go-micro\n- https://github.com/nsqio/nsq\n- https://github.com/elastic/beats\n- https://github.com/pingcap/tidb\n- https://github.com/CodisLabs/codis\n- https://github.com/baidu/bfe\n- https://github.com/caddyserver/caddy\n- https://github.com/cockroachdb/cockroach\n\n\u0026copy; [hongxi.org](http://hongxi.org) | [go.hongxi.org](http://go.hongxi.org)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjavahongxi%2Fgolab","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjavahongxi%2Fgolab","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjavahongxi%2Fgolab/lists"}