{"id":19151877,"url":"https://github.com/zhs007/finance-crawler","last_synced_at":"2026-06-19T19:02:45.192Z","repository":{"id":94328037,"uuid":"113409628","full_name":"zhs007/finance-crawler","owner":"zhs007","description":null,"archived":false,"fork":false,"pushed_at":"2018-03-20T08:57:09.000Z","size":58,"stargazers_count":1,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-03T18:46:36.185Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zhs007.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-12-07T06:01:58.000Z","updated_at":"2018-03-20T08:57:10.000Z","dependencies_parsed_at":"2023-05-05T14:47:08.352Z","dependency_job_id":null,"html_url":"https://github.com/zhs007/finance-crawler","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhs007%2Ffinance-crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhs007%2Ffinance-crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhs007%2Ffinance-crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhs007%2Ffinance-crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zhs007","download_url":"https://codeload.github.com/zhs007/finance-crawler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240236361,"owners_count":19769580,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T08:15:53.759Z","updated_at":"2026-06-14T15:30:14.965Z","avatar_url":"https://github.com/zhs007.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# finance-crawler\n\n基于```jarvis-task```的爬虫，主要用来爬取金融数据。\n\n所有的数据基本上都是全量保存的策略，所以默认的配置是数据库存储，而且做了一定的分表处理。\n\n从金融数据结构来看，我们选择的是结构化数据，MYSQL。\n\n我们尽可能的获取最原始的数据，然后根据我们需要的算法将数据转换成希望的格式。\n\n不太建议每个用户自己彻底拉取一遍数据，完整的数据量确实有点大，可以直接使用我们已经获取到的数据。\n\n------\n\n```jarvis-task```是一个分布式的任务执行系统，名字灵感来源于钢铁侠私人AI助理。\n\n数据量有些大，不太适合单机获取。\n\n------\n\n现在已经获取的数据包括：\n\n- 基金基本数据\n- 基金每日净值数据\n- 上海交易所股票基本数据\n- 深圳交易所股票基本数据\n- A股分钟级数据\n- A股日级数据\n- oanda外汇数据（分钟级）\n\n接下来应该还会增加的数据源：\n\n- 期货数据\n- 更多的外汇数据","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhs007%2Ffinance-crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzhs007%2Ffinance-crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhs007%2Ffinance-crawler/lists"}