{"id":13671058,"url":"https://github.com/youngdro/fundSpider","last_synced_at":"2025-04-27T14:33:02.860Z","repository":{"id":43898589,"uuid":"131119142","full_name":"youngdro/fundSpider","owner":"youngdro","description":"node编写的基金爬虫，可保存基金信息到数据库","archived":false,"fork":false,"pushed_at":"2018-07-07T14:15:27.000Z","size":851,"stargazers_count":108,"open_issues_count":2,"forks_count":26,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-11-11T08:43:47.626Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/youngdro.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-04-26T07:45:20.000Z","updated_at":"2024-04-26T06:38:23.000Z","dependencies_parsed_at":"2022-09-17T06:42:52.291Z","dependency_job_id":null,"html_url":"https://github.com/youngdro/fundSpider","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youngdro%2FfundSpider","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youngdro%2FfundSpider/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youngdro%2FfundSpider/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/youngdro%2FfundSpider/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/youngdro","download_url":"https://codeload.github.com/youngdro/fundSpider/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251154276,"owners_count":21544471,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T09:00:57.435Z","updated_at":"2025-04-27T14:33:02.430Z","avatar_url":"https://github.com/youngdro.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# fundSpider\n### node编写的基金爬虫，可保存基金信息到数据库\n\n具体实现过程可在[这篇文章](https://juejin.im/post/5af01d27f265da0b7f447ab6)查看\n\n启动：\n\n`npm install`\n\n`npm run start`\n\n使用express构建本地服务，启动后得到的本地数据接口如下：\n\n\u003e所有基金代码接口：\n\nhttp://localhost:1234/fetchFundCodes\n\n\u003e基金基本信息接口：\n\nhttp://localhost:1234/fetchFundInfo/:code\n\n\u003e基金净值变化数据接口：\n\nhttp://localhost:1234/fetchFundData/:code/:per\n\n\n\n得益于以下网页接口：\n\n\u003e所有基金代码网址：\n\nhttp://fund.eastmoney.com/allfund.html\n\n\u003e基金档案网页地址：\n\nhttp://fund.eastmoney.com/f10/040008.html\n\n\u003e基金净值变化接口：\n\nhttp://fund.eastmoney.com/f10/F10DataApi.aspx?type=lsjz\u0026code=040008\u0026sdate=2015-04-26\u0026edate=2018-04-26\u0026per=30\n\n所需要做的便是将这些地址爬取过滤并组合，构成方便自己调用的数据接口。\n\n在批量爬取过程中，由于node的并发特性，单次可取多条基金数据；但是经过测试，一次性并发获取所有共7000多条数据时，并发量过大导致一些请求可能会得不到返回结果，故将单次的并发量控制在1000内，再测试可完整取到所有数据。\n\n\u003e爬取的所有基金基本信息存入mongodb如下：\n\n![](https://raw.githubusercontent.com/youngdro/fundSpider/master/img/db_preview.png)\n\n---\n\n有了所有固定格式的基金数据，不放做个图表呈现出来看看。\n\n使用[Echart](http://echarts.baidu.com/) 绘制基金的增长率变化折线，模仿蚂蚁财富中可分时段查看基金净值相对变化率的形式，除此之外还可加上该时段数据的线性回归分析，根据回归线及其斜率可直观看出该时段内的基金走势；当然，数学大神还可以加以利用，做高阶的曲线拟合，甚至做数据预测。（实不相瞒，下一步的目标是做成简单的基金预测筛选推荐，保住自己的韭菜苗...）\n\n`前端侧获取基金数据后还需做简单处理` \n\n效果如下：\n\n![](https://raw.githubusercontent.com/youngdro/fundSpider/master/img/040008-1.png)\n![](https://raw.githubusercontent.com/youngdro/fundSpider/master/img/040008-2.png)\n![](https://raw.githubusercontent.com/youngdro/fundSpider/master/img/040008-3.png)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyoungdro%2FfundSpider","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyoungdro%2FfundSpider","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyoungdro%2FfundSpider/lists"}