{"id":21767950,"url":"https://github.com/mouday/freeipproxy","last_synced_at":"2026-02-16T11:02:02.057Z","repository":{"id":37063012,"uuid":"147500221","full_name":"mouday/FreeIpProxy","owner":"mouday","description":"通过抓取免费代理ip维护一个有效的proxy代理池","archived":false,"fork":false,"pushed_at":"2022-12-08T02:52:29.000Z","size":133,"stargazers_count":3,"open_issues_count":8,"forks_count":1,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-06-30T07:48:35.669Z","etag":null,"topics":["crawler","proxy","python","spider"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mouday.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-09-05T10:23:16.000Z","updated_at":"2024-01-31T13:50:44.000Z","dependencies_parsed_at":"2023-01-24T06:30:38.646Z","dependency_job_id":null,"html_url":"https://github.com/mouday/FreeIpProxy","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mouday/FreeIpProxy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mouday%2FFreeIpProxy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mouday%2FFreeIpProxy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mouday%2FFreeIpProxy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mouday%2FFreeIpProxy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mouday","download_url":"https://codeload.github.com/mouday/FreeIpProxy/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mouday%2FFreeIpProxy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29506319,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-16T09:05:14.864Z","status":"ssl_error","status_checked_at":"2026-02-16T08:55:59.364Z","response_time":115,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","proxy","python","spider"],"created_at":"2024-11-26T13:32:08.362Z","updated_at":"2026-02-16T11:02:02.040Z","avatar_url":"https://github.com/mouday.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 项目说明：\nFreeIpProxy抓取公开代理ip，并采用评分制维护代理ip池，对外提供使用接口\n\n推荐使用python3\n\n![](项目模块示意图.jpg)\n\n## 文件目录\n```\n├── README.md\n├── check.py          代理ip有效性检查\n├── config.py         全局参数配置\n├── main.py           程序入口\n├── models.py         数据库交互\n├── proxy.db          ip存储，采用sqlite，可以在models模块中替换为其他数据库\n├── proxy_source.py   代理ip抓取网站xpath配置\n├── scheduler.py      定时任务调度，抓取代理ip和检测代理ip有效性\n├── spider.py         抓取代理ip\n└── utils.py          小工具\n\n```\n\n## 采用flask提供接口：\n\n(1)获取1个代理ip\n\nGET http://127.0.0.1:5000/get\n\n按照分数值score排序，返回最高的一个值\n\n```\n{\n    id: 127,\n    ip: \"119.27.177.169\",\n    port: \"80\",\n    style: \"透明\",\n    protocol: \"HTTP\",\n    address: \"福建省厦门市\",\n    score: 14,\n    source: \"89ip代理\",\n    md5: \"9352df7efc296b1bdad2047e0c15b3a6\",\n    create_time: \"2018-09-05 15:32:49\",\n    update_time: \"2018-09-05 17:52:04\"\n}\n```\n\n(2)获取多个代理ip\n\nGET http://127.0.0.1:5000/get_list/2\n\n最后一位为数量\n\n```\n[\n    {\n        id: 127,\n        ip: \"119.27.177.169\",\n        port: \"80\",\n        style: \"透明\",\n        protocol: \"HTTP\",\n        address: \"福建省厦门市\",\n        score: 14,\n        source: \"89ip代理\",\n        md5: \"9352df7efc296b1bdad2047e0c15b3a6\",\n        create_time: \"2018-09-05 15:32:49\",\n        update_time: \"2018-09-05 17:52:04\"\n    },\n    {\n        id: 122,\n        ip: \"47.100.196.200\",\n        port: \"3128\",\n        style: \"透明\",\n        protocol: \"HTTP\",\n        address: \"上海市\",\n        score: 11,\n        source: \"89ip代理\",\n        md5: \"715de3a3237e2116e6c8a1a17f27278b\",\n        create_time: \"2018-09-05 15:32:49\",\n        update_time: \"2018-09-05 17:52:09\"\n    }\n]\n```\n(3)手动抓取\n\nGET http://127.0.0.1:5000/crawl\n\n(4)手动检查\n\nGET http://127.0.0.1:5000/check\n\n(5)手动删除\n\nGET http://127.0.0.1:5000/delete/8\n\n最后一位为返回数据中的id\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmouday%2Ffreeipproxy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmouday%2Ffreeipproxy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmouday%2Ffreeipproxy/lists"}