{"id":34574720,"url":"https://github.com/thc1006/nycu_timtable_crawler","last_synced_at":"2026-04-24T04:33:27.934Z","repository":{"id":319677025,"uuid":"1078917298","full_name":"thc1006/nycu_timtable_crawler","owner":"thc1006","description":"🎓 NYCU Course Data Crawler \u0026 Timetable System | 國立陽明交通大學課程爬蟲與選課系統 - Python web scraper for course schedules, syllabi \u0026 educational data analysis. Crawls 18K+ courses with 98% success rate. Features: interactive timetable, JSON API, Google Colab support, batch processing, resume capability.","archived":false,"fork":false,"pushed_at":"2025-10-19T14:42:10.000Z","size":4580,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-19T23:18:46.523Z","etag":null,"topics":["academic","course","course-selection","crawler","data-analysis","education","educational-data","google-colab","json-api","nycu","open-data","python","schedule","student-tools","syllabus","taiwan","timetable","university","web-automation","web-scraping"],"latest_commit_sha":null,"homepage":"https://nymu.com.tw","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thc1006.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-18T17:40:53.000Z","updated_at":"2025-10-19T14:42:13.000Z","dependencies_parsed_at":"2025-10-19T23:19:32.136Z","dependency_job_id":"1a94627f-e7eb-4ad9-9501-da47dadac49e","html_url":"https://github.com/thc1006/nycu_timtable_crawler","commit_stats":null,"previous_names":["thc1006/nycu_timtable_crawler"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/thc1006/nycu_timtable_crawler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thc1006%2Fnycu_timtable_crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thc1006%2Fnycu_timtable_crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thc1006%2Fnycu_timtable_crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thc1006%2Fnycu_timtable_crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thc1006","download_url":"https://codeload.github.com/thc1006/nycu_timtable_crawler/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thc1006%2Fnycu_timtable_crawler/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32209893,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-24T03:15:14.334Z","status":"ssl_error","status_checked_at":"2026-04-24T03:15:11.608Z","response_time":64,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["academic","course","course-selection","crawler","data-analysis","education","educational-data","google-colab","json-api","nycu","open-data","python","schedule","student-tools","syllabus","taiwan","timetable","university","web-automation","web-scraping"],"created_at":"2025-12-24T09:48:30.366Z","updated_at":"2026-04-24T04:33:27.899Z","avatar_url":"https://github.com/thc1006.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NYCU 課程爬蟲\n\n[![Python](https://img.shields.io/badge/Python-3.7+-blue.svg)](https://www.python.org)\n[![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)](LICENSE)\n[![Version](https://img.shields.io/badge/Version-v4.0/v4.1-brightgreen.svg)]()\n\n國立陽明交通大學課程資訊爬蟲 | NYCU Timetable Crawler\n\n## ✨ 功能\n\n- **雙版本**：單線程穩定版 + 4線程高性能版\n- **雙模式**：基本資訊（2-3 分鐘） / 完整綱要（40-60 分鐘）\n- **陣列格式 v2.0**：結構化時間、教室、學分等資料\n- **自動去重、智能重試、斷點續爬**\n- **66,149 門課程**覆蓋 9 個學期（723 MB）\n\n---\n\n## 📦 快速開始\n\n### 安裝 \u0026 使用\n\n```bash\n# 安裝依賴\npip install -r requirements.txt\n\n# 方式 1: 單線程版本（穩定性優先）\npython nycu_crawler.py                              # 基本資訊（預設 114-1）\npython nycu_crawler.py --year 113 --semester 2     # 爬取特定學期\npython nycu_crawler.py --outline                    # 爬取完整綱要\n\n# 方式 2: 4線程版本（性能優先，推薦）\npython nycu_crawler_multithreaded.py                # 基本資訊\npython nycu_crawler_multithreaded.py --outline      # 完整綱要\npython nycu_crawler_multithreaded.py --threads 8   # 自訂線程數\n\n# 範例腳本\npython examples/analyze_statistics.py    # 統計分析\npython examples/check_conflicts.py       # 衝堂檢測\npython examples/search_courses.py        # 課程搜尋\n```\n\n---\n\n## ⚙️ 命令行參數\n\n| 參數 | 說明 | 預設 |\n|------|------|------|\n| `--year` | 學年度 (100-200) | 114 |\n| `--semester` | 學期 (1=上, 2=下) | 1 |\n| `--outline` | 爬取完整課程綱要 | False |\n| `--threads` | 線程數 (1-16，4線程版本) | 4 |\n| `--timeout` | 超時秒數 (4線程版本) | 600 |\n| `--help` | 顯示幫助 | - |\n| `--version` | 顯示版本 | - |\n\n---\n\n## 📈 性能 \u0026 精確度\n\n| 項目 | 基本資訊 | 完整綱要 |\n|------|---------|---------|\n| **執行時間** | 2-3 分鐘 | 40-60 分鐘 |\n| **檔案大小** | 5-6 MB | 43-63 MB |\n| **解析成功率** | 100% | \u003e95% |\n\n- 單個學期：7,000-8,000 門課程\n- 全部 9 學期：66,149 門課程，723 MB\n- 去重率：100% | 時間表解析：\u003e99%\n\n---\n\n## 📁 專案結構\n\n```\nnycu_timtable_crawler/\n├── nycu_crawler.py                  # 單線程版本\n├── nycu_crawler_multithreaded.py    # 4線程版本\n├── examples/\n│   ├── analyze_statistics.py        # 統計分析\n│   ├── check_conflicts.py          # 衝堂檢測\n│   └── search_courses.py           # 課程搜尋\n├── notebooks/                       # Jupyter Notebook\n├── course_data/                     # 爬蟲數據 (723 MB)\n│   ├── basic/                       # 基本資訊 (54 MB)\n│   └── with_outline/                # 完整綱要 (676 MB)\n└── requirements.txt, LICENSE, README.md\n```\n\n---\n\n## 📊 資料格式 (v2.0)\n\nJSON 包含 `metadata` 和 `courses` 陣列：\n\n```json\n{\n  \"metadata\": {\n    \"semester\": \"114-1\",\n    \"total_courses\": 8028,\n    \"crawler_version\": \"4.0\",\n    \"data_format_version\": \"2.0\"\n  },\n  \"courses\": [\n    {\n      \"id\": \"515002\",\n      \"name\": \"微分方程\",\n      \"teacher\": \"楊春美\",\n      \"credit\": 3.0,\n      \"type\": \"必修\",\n      \"enrollment\": {\"limit\": 55, \"current\": 66},\n      \"schedule\": [\n        {\n          \"day\": 1,\n          \"periods\": [3, 4],\n          \"time_start\": \"10:10\",\n          \"classroom\": \"EE102\"\n        }\n      ]\n    }\n  ]\n}\n```\n\n**改進**：陣列格式、數字型別、結構化時間、完整 metadata、RESTful API 標準\n\n---\n\n## 🚀 常用場景\n\n| 需求 | 建議命令 | 耗時 |\n|------|---------|------|\n| 快速取得課表 | `python nycu_crawler_multithreaded.py` | 2-3 分鐘 |\n| 穩定爬取 | `python nycu_crawler.py` | 2-3 分鐘 |\n| 完整綱要 | `python nycu_crawler_multithreaded.py --outline` | 40-50 分鐘 |\n| 特定學期 | `python nycu_crawler.py --year 113 --semester 2` | 2-3 分鐘 |\n| 統計分析 | `python examples/analyze_statistics.py` | - |\n| 衝堂檢測 | `python examples/check_conflicts.py` | - |\n\n---\n\n## 💡 常見問題\n\n**Q: 單線程 vs 4線程？** → 單線程更穩定，4線程更快（特別是綱要爬取）\n\n**Q: 爬取綱要需要多久？** → 單線程 50-60 分鐘，4線程 40-50 分鐘\n\n**Q: 如何爬取特定學期？**\n```bash\npython nycu_crawler.py --year 113 --semester 2  # 113年第2學期\n```\n\n**Q: 完整綱要包含什麼？** → 課程描述、先修科目、評分方式、教科書、16-18週授課進度\n\n**Q: 線程數越多越好？** → 建議 2-8，過多可能被限流。使用 `--threads 8`\n\n**Q: 資料更新頻率？** → 建議每學期開學前 1-2 週爬取一次\n\n---\n\n## 🔧 技術細節\n\n**時間解析**：原始格式 `M34W2-EE102[GF]` → 結構化 `{day:1, periods:[3,4], time_start:\"10:10\", classroom:\"EE102\"}`\n\n**星期代碼**：M/T/W/R/F/S/U = 一/二/三/四/五/六/日\n\n**節次對應**：1=08:00, 2=09:00, 3=10:10, 4=11:10, 5=13:20, 6=14:20, 7=15:30, 8=16:30, 9=17:30, a=18:25, b=19:20, c=20:15, d=21:10\n\n**爬取策略**：類型→類別→學院→系所→課程、智能去重、指數退避重試、備選HTML解析、斷點續爬\n\n---\n\n## 📚 範例腳本\n\n| 腳本 | 功能 |\n|------|------|\n| `analyze_statistics.py` | 統計學分分布、課程類型、授課語言、熱門時段、額滿課程等 |\n| `check_conflicts.py` | 檢測兩門課程是否衝突、檢查選課清單衝堂 |\n| `search_courses.py` | 按名稱、教師、學分、星期搜尋課程 |\n\n---\n\n## 📝 版本歷史\n\n| 版本 | 發布日期 | 主要功能 |\n|------|---------|---------|\n| v4.1 | 2025-01-20 | 4線程版本、ThreadPoolExecutor、Thread-safe、性能 +20-30% |\n| v4.0 | 2025-10-20 | 陣列格式v2.0、結構化時間、argparse參數、metadata |\n| v3.0 | - | 斷點續爬、錯誤重試、進度顯示、網絡優化 |\n| v2.0 | - | 完整綱要支援、批次爬取、多學期自動化 |\n| v1.0 | - | 基本課程資訊爬取 |\n\n---\n\n## 📜 授權與貢獻\n\n**授權**：Apache License 2.0 | [詳見 LICENSE](LICENSE)\n\n**聯絡**：hctsai@linux.com | [GitHub Issues](https://github.com/thc1006/nycu_timtable_crawler/issues)\n\n**相關文件**：[IMPLEMENTATION_REPORT.md](IMPLEMENTATION_REPORT.md)\n\n---\n\n## 📊 項目統計\n\n| 項目 | 統計 |\n|------|------|\n| Python 源碼 | 2,052 行（1,735 主程式 + 317 範例） |\n| 課程數據 | 66,149 門課程（9 個學期） |\n| 數據大小 | 723.4 MB（基本 54 MB + 綱要 676 MB） |\n| 環境等級 | Production Grade ✓ |\n\n**最後更新**：2025-10-20 | v4.0/v4.1 | v2.0 格式\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthc1006%2Fnycu_timtable_crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthc1006%2Fnycu_timtable_crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthc1006%2Fnycu_timtable_crawler/lists"}