{"id":43141237,"url":"https://github.com/jser/dataset","last_synced_at":"2026-01-31T22:39:50.877Z","repository":{"id":32221863,"uuid":"131940335","full_name":"jser/dataset","owner":"jser","description":"JSer.infoのデータセットや処理ライブラリ","archived":false,"fork":false,"pushed_at":"2025-05-06T10:38:51.000Z","size":4680,"stargazers_count":4,"open_issues_count":4,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-10-22T13:37:19.987Z","etag":null,"topics":["api","jser","stat","statistics"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jser.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":"azu"}},"created_at":"2018-05-03T04:08:03.000Z","updated_at":"2025-10-04T16:37:59.000Z","dependencies_parsed_at":"2024-02-21T04:30:48.856Z","dependency_job_id":"7e784d50-4591-4abd-9930-d776c813d241","html_url":"https://github.com/jser/dataset","commit_stats":{"total_commits":273,"total_committers":3,"mean_commits":91.0,"dds":"0.10989010989010994","last_synced_commit":"97fb6ce2b877230560441f44a4f3f3a2bd701513"},"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"purl":"pkg:github/jser/dataset","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jser%2Fdataset","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jser%2Fdataset/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jser%2Fdataset/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jser%2Fdataset/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jser","download_url":"https://codeload.github.com/jser/dataset/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jser%2Fdataset/sbom","scorecard":{"id":538124,"data":{"date":"2025-08-11","repo":{"name":"github.com/jser/dataset","commit":"96540f87f3540860db57d83389755415c9cae6ff"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":4.2,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Code-Review","score":0,"reason":"Found 0/22 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Pinned-Dependencies","score":10,"reason":"all dependencies are pinned","details":["Info:   2 out of   2 GitHub-owned GitHubAction dependencies pinned","Info:   1 out of   1 third-party GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/test.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Security-Policy","score":9,"reason":"security policy file detected","details":["Info: security policy file detected: github.com/jser/.github/SECURITY.md:1","Info: Found linked content: github.com/jser/.github/SECURITY.md:1","Warn: One or no descriptive hints of disclosure, vulnerability, and/or timelines in security policy","Info: Found text in security policy: github.com/jser/.github/SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 17 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":5,"reason":"5 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-v6h2-p8h4-qcjw","Warn: Project is vulnerable to: GHSA-fjxv-7rqg-78g4","Warn: Project is vulnerable to: GHSA-rp65-9cf3-cjxr","Warn: Project is vulnerable to: GHSA-52f5-9888-hmc6","Warn: Project is vulnerable to: GHSA-w5p7-h5w8-2hfq"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-20T07:31:28.202Z","repository_id":32221863,"created_at":"2025-08-20T07:31:28.202Z","updated_at":"2025-08-20T07:31:28.202Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28958348,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-31T22:20:19.638Z","status":"ssl_error","status_checked_at":"2026-01-31T22:18:07.061Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","jser","stat","statistics"],"created_at":"2026-01-31T22:39:46.034Z","updated_at":"2026-01-31T22:39:50.870Z","avatar_url":"https://github.com/jser.png","language":"HTML","funding_links":["https://github.com/sponsors/azu"],"categories":[],"sub_categories":[],"readme":"# DataSet [![Actions Status: test](https://github.com/jser/dataset/workflows/test/badge.svg)](https://github.com/jser/dataset/actions?query=workflow%3A\"test\")\n\nJSer.infoのデータセットをまとめたmonorepoです。\n\nJSer.infoで収集しているデータや記事を書くために利用している統計ライブラリなどが公開されています。\n\n## データの種類\n\nそれぞれのデータは[Creative Commons — Attribution 4.0 International — CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)の元で利用できます。\n\n- Item: 紹介するサイトのこと\n    - 1 Item = 1 サイト\n    - すべてのデータのoriginとなるものです\n    - サイトごとにタイトル、URL、登録した日付、タグなどが含まれています\n    - API: \u003chttps://jser.info/source-data/items.json\u003e\n- Post: JSer.infoに投稿される記事のこと\n    - 1 Post = 1 記事\n    - それぞれの記事のタイトル、URL、タグ、日付などが含まれます\n    - [@jser/stat][]を使うことでItemとPostを元に指定したサイトが紹介された記事を検索できます\n    - API: \u003chttps://jser.info/posts.json\u003e\n- Post Item: JSer.infoに投稿された記事中のItem(サイト)のこと\n    - 1 Post Item = 1 サイト\n    - 基本的にはItemと同じだが、Post ItemはPost(記事)におけるカテゴリ（ヘッドラインなど）が含まれます\n    - カテゴリの種類は [@jser/post-parser][] を参照してください\n    - Itemを元に投稿時に編集している場合などもあるため、ItemとPost Itemは必ずしも一致するわけではありません\n    - 制限: カテゴリ区別が付けられたのは[2014-08-03](https://jser.info/2014/08/03/renewal/)からであるため、それ以前のデータは含まれない\n    - Postにはすべての記事は含まれるがPost Itemのデータは含まれていない\n    - API: \u003chttps://jser.info/public/data/post-details.json\u003e\n\nデータは個別だと扱いにくい場合などがあります。\nまた、時期によって特定のプロパティが欠損してる場合もあるため、後述する分析ライブラリなどのHigh LevelなAPIを利用することを推奨します。\n\n[@jser/data-fetcher][]でデータとして取得し、[@jser/stat][]などの分析ライブラリに与えて利用するとある程度正規化されます。\n\n統計データの閲覧やCSVデータの取得なら[JSer.info Data Dashboard](https://jser.info/data-dashboard/)も利用できます。\n\n## Fetcher\n\n[@jser/data-fetcher][] を使うことでデータを取得できます。\n\n```ts\nimport { fetchItems, fetchPostDetails, fetchPosts } from \"@jser/data-fetcher\";\n// Item\n// https://github.com/jser/source-data\nawait fetchItems();\n// Post\n// https://github.com/jser/jser.github.io/blob/develop/posts.json\nawait fetchPosts();\n// Post Details include Post Item\n// https://github.com/jser/jser.github.io/blob/develop/_tools/generate-post-details-json.js\nawait fetchPostDetails();\n```\n\n## 分析ライブラリ\n\n### [@jser/classifier-item-category][]\n\nJSer.infoの学習データを使い、渡した文字列がJSer.infoではどのカテゴリに分類されるかを推論します。\nJSer.infoで記事を投稿する際のカテゴリ分類に利用しています。\n\n### [@jser/stat][]\n\nJSer.infoのデータを使った統計の前処理ライブラリです。\nItemとPostのデータを正規化し、指定した日付のItemの取得、Postの取得、関連する記事の検索などが行えます。\n\nJSer.infoでは次のような場所で利用しています。\n\n* 記事の関連記事探索\n* [jser/data-dashboard: JSer.info Data Dashboard.](https://github.com/jser/data-dashboard \"jser/data-dashboard: JSer.info Data Dashboard.\")\n* [jser/trends: JSer.info トレンドビューア](https://github.com/jser/trends \"jser/trends: JSer.info トレンドビューア\")\n* [JSer.info Watch List](https://jser.info/watch-list/)\n* [JSer.info Product Name API](https://github.com/jser/product-name)\n* [jser/mcp: An MCP Server for JSer.info](https://github.com/jser/mcp)\n\n## Changelog\n\nSee [Releases page](https://github.com/jser/dataset/releases).\n\n## Running tests\n\n    pnpm install\n    pnpm test\n\n## Contributing\n\nPull requests and stars are always welcome.\n\nFor bugs and feature requests, [please create an issue](https://github.com/jser/dataset/issues).\n\n1. Fork it!\n2. Create your feature branch: `git checkout -b my-new-feature`\n3. Commit your changes: `git commit -am 'Add some feature'`\n4. Push to the branch: `git push origin my-new-feature`\n5. Submit a pull request :D\n\n## Author\n\n- [github/azu](https://github.com/azu)\n- [twitter/azu_re](https://twitter.com/azu_re)\n\n## License\n\n### Code\n\nMIT © azu\n\n### Dataset\n\nCC-BY [![CC-BY](https://i.creativecommons.org/l/by/4.0/88x31.png)](http://creativecommons.org/licenses/by/4.0/)\n\n[@jser/post-parser]: packages/@jser/post-parser\n[@jser/data-fetcher]: packages/@jser/data-fetcher\n[@jser/classifier-item-category]: packages/@jser/classifier-item-category\n[@jser/stat]: packages/@jser/stat\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjser%2Fdataset","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjser%2Fdataset","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjser%2Fdataset/lists"}