{"id":35132317,"url":"https://github.com/adminlove520/nsfw-collector","last_synced_at":"2026-05-12T04:00:44.384Z","repository":{"id":329048143,"uuid":"1116681382","full_name":"adminlove520/NSFW-collector","owner":"adminlove520","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-08T02:07:25.000Z","size":650861,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-08T04:39:06.136Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adminlove520.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-15T08:23:33.000Z","updated_at":"2026-01-21T10:10:55.000Z","dependencies_parsed_at":"2026-05-12T04:00:38.570Z","dependency_job_id":null,"html_url":"https://github.com/adminlove520/NSFW-collector","commit_stats":null,"previous_names":["adminlove520/nsfw-collector"],"tags_count":146,"template":false,"template_full_name":null,"purl":"pkg:github/adminlove520/NSFW-collector","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adminlove520%2FNSFW-collector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adminlove520%2FNSFW-collector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adminlove520%2FNSFW-collector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adminlove520%2FNSFW-collector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adminlove520","download_url":"https://codeload.github.com/adminlove520/NSFW-collector/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adminlove520%2FNSFW-collector/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32922890,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-11T17:09:15.040Z","status":"online","status_checked_at":"2026-05-12T02:00:06.338Z","response_time":102,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-28T06:35:24.675Z","updated_at":"2026-05-12T04:00:44.349Z","avatar_url":"https://github.com/adminlove520.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 涩涩-collector\n\n一个涩涩采集器，支持图片和小说两种采集模式，并提供每日模式选项。\n\n## 功能特性\n\n### 采集模式\n- **图片模式**：采集指定版块的图片内容，支持子版块：\n  - gif動圖 (ID: 33)\n  - 亞洲激情 (ID: 29)\n  - 露出清纯 (ID: 30)\n  - 歐美性愛 (ID: 31)\n  - 卡通動漫 (ID: 32)\n\n- **小说模式**：采集指定版块的小说内容，支持子版块：\n  - 人妻熟女 (ID: 24)\n  - 經典激情 (ID: 28)\n  - 家庭亂倫 (ID: 26)\n  - 武俠玄幻 (ID: 35)\n  - 學生校園 (ID: 27)\n\n### 每日模式\n- 支持 `--daily` 选项，仅采集当日发布的内容\n- 当日数据保存在 `./picture/daily_YYYY-MM-DD/` 和 `./novel/daily_YYYY-MM-DD/` 目录\n- 自动根据帖子标题中的日期标识 `[MM-DD]` 进行筛选\n\n### 配置灵活性\n- 支持通过 `config.yaml` 本地配置\n- 支持通过 GitHub Workflow 输入参数配置\n- 支持多个备选域名配置 (`site_domain.yaml`)\n- 可配置请求头、超时、延迟和重试次数\n\n## 安装依赖\n\n```bash\npip install requests beautifulsoup4 pyyaml\n```\n\n## 使用方法\n\n### 命令行运行\n\n```bash\n# 运行图片采集模式\npython main.py --mode picture\n\n# 运行小说采集模式\npython main.py --mode novel\n\n# 运行每日图片采集模式\npython main.py --mode picture --daily\n\n# 运行每日小说采集模式\npython main.py --mode novel --daily\n\n# 指定配置文件\npython main.py --mode picture --config my_config.yaml\n```\n\n### 配置文件说明\n\n#### config.yaml\n\n```yaml\ncrawl:\n  max_pages: 10         # 每个版块最大爬取页数\n  retry_times: 3        # 请求失败重试次数\ncrawl_mode: picture     # 默认采集模式\n\n# 小说版块配置\nnovel_forums:\n- id: 24\n  name: 人妻熟女\n- id: 28\n  name: 經典激情\n- id: 26\n  name: 家庭亂倫\n- id: 35\n  name: 武俠玄幻\n- id: 27\n  name: 學生校園\n\n# 图片版块配置\npicture_forums:\n- id: 33\n  name: gif動圖\n- id: 29\n  name: 亞洲激情\n- id: 30\n  name: 露出清纯\n- id: 31\n  name: 歐美性愛\n- id: 32\n  name: 卡通動漫\n\nrequest:\n  delay: 1              # 请求延迟（秒）\n  headers:              # 请求头\n    User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36\n  proxies:              # 代理配置\n    http: null\n    https: null\n  timeout: 10           # 请求超时（秒）\n\nsave_paths:\n  novel: ./novel        # 小说保存路径\n  picture: ./picture    # 图片保存路径\n\nsite_domain: wm.wmhuu.com  # 网站域名\n```\n\n#### site_domain.yaml（可选）\n\n```yaml\nsite_domains:\n  - wm.wmhuu.com\n  - backup.wmhuu.com\n```\n\n## GitHub Actions 工作流\n\n### 手动触发\n\n通过 GitHub Actions 手动触发工作流时，可以选择以下参数：\n\n- **crawl_mode**：采集模式，可选值：picture、novel\n- **daily_mode**：是否仅采集当日数据，默认值：true\n\n### 定时执行\n\n工作流配置了每日凌晨自动执行，默认使用每日模式采集图片。\n\n## 项目结构\n\n```\ngetSeSe/\n├── main.py                # 主程序入口\n├── config.yaml            # 主配置文件\n├── site_domain.yaml       # 域名配置文件\n├── .github/workflows/\n│   └── crawler.yml        # GitHub Actions 工作流\n├── utils/\n│   ├── request.py         # 请求处理模块\n│   ├── parser.py          # HTML 解析模块\n│   └── saver.py           # 内容保存模块\n├── picture/               # 图片保存目录\n├── novel/                 # 小说保存目录\n└── README.md              # 项目说明文档\n```\n\n## 注意事项\n\n1. 请遵守网站的 robots.txt 规则，合理设置爬取间隔\n2. 大量爬取可能导致 IP 被封禁，建议使用代理\n3. 本程序仅供学习和研究使用，请勿用于商业用途\n4. 定期检查并更新域名配置，以确保爬虫正常运行\n\n## 许可证\n\nMIT License\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadminlove520%2Fnsfw-collector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadminlove520%2Fnsfw-collector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadminlove520%2Fnsfw-collector/lists"}