{"id":24674024,"url":"https://github.com/taoget/livetv_mining","last_synced_at":"2025-06-17T23:39:31.834Z","repository":{"id":94737004,"uuid":"54558628","full_name":"taogeT/livetv_mining","owner":"taogeT","description":"直播网站数据采集","archived":false,"fork":false,"pushed_at":"2017-06-27T07:42:49.000Z","size":518,"stargazers_count":213,"open_issues_count":0,"forks_count":94,"subscribers_count":24,"default_branch":"master","last_synced_at":"2025-05-09T00:06:06.730Z","etag":null,"topics":["flask","python","python3","scrapy","vue","webpack"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/taogeT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2016-03-23T12:38:10.000Z","updated_at":"2025-04-06T19:16:30.000Z","dependencies_parsed_at":"2023-03-23T07:41:51.336Z","dependency_job_id":null,"html_url":"https://github.com/taogeT/livetv_mining","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/taogeT/livetv_mining","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taogeT%2Flivetv_mining","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taogeT%2Flivetv_mining/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taogeT%2Flivetv_mining/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taogeT%2Flivetv_mining/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/taogeT","download_url":"https://codeload.github.com/taogeT/livetv_mining/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taogeT%2Flivetv_mining/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260458909,"owners_count":23012506,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["flask","python","python3","scrapy","vue","webpack"],"created_at":"2025-01-26T11:14:45.671Z","updated_at":"2025-06-17T23:39:26.817Z","avatar_url":"https://github.com/taogeT.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"### 关于项目\n***\n\n#### 介绍\n\n项目实现**爬虫**、**数据挖掘/分析**等功能。目前主要是扫描直播站点收集数据，后续会开发相关数据分析模块。\n\n架构采用前后端分离，通过RESTFul API通信。前端Vue+Webpack，后端Flask。\n\n文件结构分为三部分，每个部分可以单独提供服务：\n\n* frontend 前端代码，使用Vue+Webpack开发管理。\n* server   后端RESTFul服务，使用Flask搭建提供API。\n* crawler  爬虫服务，使用框架scrapy对直播站点爬取数据。\n\n使用中有问题或更新建议，欢迎提issue或用以下联系方式跟我交流：\n\n* 邮件: zwtzjd@gmail.com\n* QQ: 3084582097\n\n#### 历程\n\n* 2017-05-10 开发新爬虫火猫。\n* 2017-04-24 全民收集开发完成。\n* 2017-04-22 熊猫收集开发完成。\n* 2017-04-20 斗鱼收集开发完成。\n* 2017-04-15 准备开发收集站点房间：关注数、公告、上一次开播时间、打赏值、开播当天人气中位数。\n* 2017-04-01 变更LICENSE：GPL -\u003e Apache。\n* 2017-03-20 开发新爬虫全民。\n* 2017-03-03 开发新爬虫龙珠。\n* 2017-03-01 恢复爬虫熊猫、战旗。\n* 2017-01-07 完成全部功能迁移，前后端分离完成。\n* 2016-12-24 引入vuex进行state和权限管理。\n* 2016-12-15 引入vue-resource，主模块(Main)完成分离。\n* 2016-11-25 研究流行前端框架和开发方式，确定使用webpack开发。\n* 2016-11-20 重新设计结构，进行前后端分离。\n* 2016-11-01 修正改造搜索功能，需要登录后才能使用。\n* 2016-10-29 完成渲染初步改造，已实现调用REST API进行访问读取。爬虫运行间隔20分钟。\n* 2016-10-20 引入Vue.js，改造前端渲染方式，减轻服务器压力。开发完成简单 [Flask-Vue](https://github.com/taogeT/flask-vue) 插件。\n* 2016-10-15 引入Scrapy重构爬虫，精简爬虫步骤。仅保留斗鱼、BiliBli爬虫。\n\n##### 2016-08-15 v1.0版本开发结束，代码保存在分支v1.0\n\n### 捐助开发者\n***\n\n如果你喜欢我的作品 or 站点对你有所帮助，希望能支持我一下。\n\n| 微信 | 支付宝 |\n| ------ | ------ |\n| \u003cimg width=\"80%\" src=\"http://ww4.sinaimg.cn/large/77ef2ab1gw1f2ne2zgy4zj20bt0bqq4s.jpg\"\u003e | ![alipay](http://ww3.sinaimg.cn/mw690/77ef2ab1gw1f2ne2zc10cj209a09bdgz.jpg) |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaoget%2Flivetv_mining","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftaoget%2Flivetv_mining","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaoget%2Flivetv_mining/lists"}