{"id":18895870,"url":"https://github.com/maicss/1024img","last_synced_at":"2026-05-03T09:36:37.582Z","repository":{"id":110162827,"uuid":"95867598","full_name":"maicss/1024img","owner":"maicss","description":"1024 image nodejs crawler","archived":false,"fork":false,"pushed_at":"2018-05-19T02:28:29.000Z","size":98,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-12-31T07:45:31.803Z","etag":null,"topics":["1024","crawler","nodejs"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/maicss.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-06-30T08:34:11.000Z","updated_at":"2024-01-19T02:34:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"6980f0dd-3e7e-4690-a28e-22349476099c","html_url":"https://github.com/maicss/1024img","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maicss%2F1024img","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maicss%2F1024img/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maicss%2F1024img/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maicss%2F1024img/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/maicss","download_url":"https://codeload.github.com/maicss/1024img/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239870515,"owners_count":19710736,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["1024","crawler","nodejs"],"created_at":"2024-11-08T08:30:55.841Z","updated_at":"2026-02-28T01:30:18.425Z","avatar_url":"https://github.com/maicss.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 1024img\n\n\u003e 闲着没事写的爬虫\n\n主要爬两个页面，分别是[达盖尔的旗帜]和[新时代的我们]。\n\n\u003e 360浏览器有个功能，下载本页面所有图片，而且图片也能设置大小等规则，如果不是大批量的下载，这个够用了。使用我写的这个反而更麻烦，嗯嗯。chrome插件我没找。\n\n## 项目进度\n\n- [ x ] 基本功能完成\n- [ x ] 达盖尔的旗帜 测试\n- [ ] 新时代的我们 测试\n- [ ] 数据库的读写\n\n## 注\n\n你们应该都知道这个爬虫是经过代理的，没有代理，你们就看代码好了😝\n\n\n## 数据\n\n返回的数据结构：可以直接看`Interfaces.ts`\n\n```json\n{\n    \"postName\": \" [原创][[cl分享团出品]xxx[19P]\",\n    \"postTime\": \"2018-03-23 21:04\",\n    \"postUrl\": \"http://t66y.com/htm_data/xxx.html\",\n    \"highlight\": true,\n    \"done\": true,\n    \"images\": [\n        {\n            \"url\": \"http://s6tu.com/images/2018/03/11/xxx.jpg\",\n            \"index\": 1,\n            \"id\": \"96w6q01I1\",\n            \"downloaded\": false,\n            \"retryTime\": 0\n        }\n    ]\n}\n```\n\n## 其他\n\n主要是想试试node的爬虫，和`async`\u0026`await`的使用。\n\n数据库不是必要的，但是也写了，用的`mongodb`。这个比较有趣，因为官方drive有ES6的教程，使用的是co，也就是generator，现在有了原生的`async`\u0026`await`（其实还是generator）写起来更方便了。\n\n现在（node8）也自带了`promisify`又省了一个库，爽。\n\n大家选择node的原因无非是：\n\n- 会javascript\n- 基于事件的异步执行\n- 轻量\n\n但是这里主要的目的并不是爬虫，所以异步并发的反而不是优点，所以全是同步的操作~~开着坦克不能压坏路的感觉~~。其实这种场景用python，一个接一个的爬，什么也不用考虑，真的比node爽多了。\n\nHappy watching~~~","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaicss%2F1024img","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaicss%2F1024img","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaicss%2F1024img/lists"}