{"id":13385035,"url":"https://github.com/luyishisi/Anti-Anti-Spider","last_synced_at":"2025-03-13T10:31:26.627Z","repository":{"id":41274841,"uuid":"73654812","full_name":"luyishisi/Anti-Anti-Spider","owner":"luyishisi","description":"越来越多的网站具有反爬虫特性，有的用图片隐藏关键数据，有的使用反人类的验证码，建立反反爬虫的代码仓库，通过与不同特性的网站做斗争（无恶意）提高技术。（欢迎提交难以采集的网站）（因工作原因，项目暂停） ","archived":false,"fork":false,"pushed_at":"2021-10-17T20:14:19.000Z","size":150743,"stargazers_count":7283,"open_issues_count":27,"forks_count":2173,"subscribers_count":449,"default_branch":"master","last_synced_at":"2025-02-27T13:53:01.119Z","etag":null,"topics":["geek","python","spider"],"latest_commit_sha":null,"homepage":"https://www.urlteam.cn","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/luyishisi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-11-14T01:18:11.000Z","updated_at":"2025-02-25T08:05:16.000Z","dependencies_parsed_at":"2022-07-13T15:59:46.584Z","dependency_job_id":null,"html_url":"https://github.com/luyishisi/Anti-Anti-Spider","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luyishisi%2FAnti-Anti-Spider","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luyishisi%2FAnti-Anti-Spider/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luyishisi%2FAnti-Anti-Spider/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luyishisi%2FAnti-Anti-Spider/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/luyishisi","download_url":"https://codeload.github.com/luyishisi/Anti-Anti-Spider/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243386079,"owners_count":20282688,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["geek","python","spider"],"created_at":"2024-07-30T11:01:02.720Z","updated_at":"2025-03-13T10:31:24.262Z","avatar_url":"https://github.com/luyishisi.png","language":"Python","funding_links":[],"categories":["Python (1887)","Python","HTML","Crawler"],"sub_categories":[],"readme":"## 基于CNN的验证码图片识别\n### 简介\n\t本项目采用alexnet模型和letnet模型，可根据实际需要选择(在train_model.py中的train函数修改即可)95.5%\n### 作者有话说\n\t不知不觉这个git库伴随我从16到到20年，带给我自己最棒的一段人生旅程，\n\t整理了这份文档，希望任何想学习图片识别，玩玩卷积神经网络的同学可以最便捷的上手体验。\n\t请谨慎使用技术，仅支持学习，不支持任何黑灰产相关\n\t可参看：https://www.urlteam.cn/?p=1893 https://www.urlteam.cn/?p=1406\n\t原先的Anti-Anti-Spider 全部内容移动到 原Anti-Anti-Spider 目录下\n\t有何疑问可邮件 543429245@qq.com 咨询\n\t模型文件下载 如果出现无法解压，可以使用：\n\thttps://www.urlteam.cn/%E5%8F%AF%E7%94%A8%E8%AE%AD%E7%BB%83%E9%9B%86%E4%B8%8E%E8%AE%AD%E5%A5%BD%E7%9A%84%E6%A8%A1%E5%9E%8B.zip\n\n#### Alexnet 模型结构\n\n![](src/READMEIMG2.PNG)\n\n根据验证码的复杂度不同，训练的时间也会有较大的不同\n![](src/READMEIMG1.PNG)\n\n###  使用方法\n\t1.开始训练样本前，修改conf/config.json\n\t2.将预处理过的数据集分成验证集和训练集，放到sample目录下\n\t3.运行train_model.py开始训练，训练完成的模型保存至model_result中\n\t4.将训练好的模型放置model_result，运行cnn_models/recognition.py，选定验证码，即可看到模型效果\n### 环境配置\nTensorFlow CPU版本安装：`pip install tensorflow==1.9.0`\nTensorFlow GPU版本安装：`pip install tensorflow-gpu==1.9.0`\nGUP版本的安装比较麻烦，需要安装CUDA和cuDNN才能使tensorflow调动GPU\n下图为TensorFlow，Python，CUDA与cuDNN之间的版本对应关系：\n![](./src/README_IMG0.PNG)\nCUDA与cuDNN安装过程主要有两步：\n\n1. 到官网下载CUDA并安装\n2. 将cuDNN解压，复制到CUDA安装目录下\n这里提供两个文件的链接：\nCUDA：`https://developer.nvidia.com/cuda-toolkit-archive`\ncuDNN：`https://developer.nvidia.com/rdp/cudnn-archive`\n更具体的安装过程度娘可帮你轻松解决(linux,windows这两步的操作方法各不相同)\n### 项目结构\n```\n├─cnn_models\n│  ├─cnn_model.py\t\t# CNN网络类\n│  └─recognition.py\t\t# 验证训练结果\n├─conf\n│  └─config.json\t\t# 配置文件\n├─logs\t\t\t# 模型训练日志\n├─model_result\t# 模型保存地址\n│  └─1040\t\t# 一套训练完成的验证码训练集及对应模型\n├─sample\n│  ├─test\t\t# 训练集（训练集与验证集一般是对总数据集9:1分割）\n│  └─train\t\t# 验证集\n├─src\t\t\t# 配置环境所需的工具，可根据自身情况到网上下载\n├─train_model.py\t\t# 训练程序\n└─verify_sample.py\t\t# 制作数据集（打标签加图片预处理）\n```\n### 图片预处理\n+ 为验证码图片打上标签，如：\n![](./src/1040_2019-10-13_10_1092.jpg)\n命名为1040_2019-10-13_10_1092.jpg，1092为标签，其余为附加信息，可根据自己需要更改，用`_`分割即可\n+ 由于模型输入要求输入必须为227*227，所有需要调整图片形状，verify_sample.py中提供有工具函数\n### 注意事项\n\talexnet输入必须为227*227的图片，所有图片预处理时可通过PIL中的函数线性转换图片形状，或者缩放后粘贴到227*227的背景中。\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluyishisi%2FAnti-Anti-Spider","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fluyishisi%2FAnti-Anti-Spider","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluyishisi%2FAnti-Anti-Spider/lists"}