{"id":21246908,"url":"https://github.com/mitevpi/poli-parse","last_synced_at":"2026-05-13T23:38:03.300Z","repository":{"id":35010716,"uuid":"192122315","full_name":"mitevpi/poli-parse","owner":"mitevpi","description":"Political news scraping \u0026 NLP parsing from web pages.","archived":false,"fork":false,"pushed_at":"2023-01-04T21:54:44.000Z","size":3597,"stargazers_count":0,"open_issues_count":12,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-08-29T22:00:22.239Z","etag":null,"topics":["data","election","javascript","library","module","nlp","npm","package","parse","politics","scrape","sentiment"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mitevpi.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-06-15T20:44:06.000Z","updated_at":"2021-03-19T12:38:54.000Z","dependencies_parsed_at":"2023-01-15T11:57:10.481Z","dependency_job_id":null,"html_url":"https://github.com/mitevpi/poli-parse","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mitevpi/poli-parse","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitevpi%2Fpoli-parse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitevpi%2Fpoli-parse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitevpi%2Fpoli-parse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitevpi%2Fpoli-parse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mitevpi","download_url":"https://codeload.github.com/mitevpi/poli-parse/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitevpi%2Fpoli-parse/sbom","scorecard":{"id":650360,"data":{"date":"2025-08-11","repo":{"name":"github.com/mitevpi/poli-parse","commit":"f62d8629405981a087b3650be203876da85cc980"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":1.7,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":0,"reason":"51 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-968p-4wvh-cqc8","Warn: Project is vulnerable to: GHSA-67hx-6x53-jw92","Warn: Project is vulnerable to: GHSA-whgm-jr23-g3j9","Warn: Project is vulnerable to: GHSA-93q8-gq69-wqmw","Warn: Project is vulnerable to: GHSA-cph5-m8f7-6c5x","Warn: Project is vulnerable to: GHSA-wf5p-g6vw-rhxx","Warn: Project is vulnerable to: GHSA-jr5f-v2jv-69x6","Warn: Project is vulnerable to: GHSA-v6h2-p8h4-qcjw","Warn: Project is vulnerable to: GHSA-grv7-fg5c-xmjg","Warn: Project is vulnerable to: GHSA-w8qv-6jwh-64r5","Warn: Project is vulnerable to: GHSA-wg6g-ppvx-927h","Warn: Project is vulnerable to: GHSA-3xgq-45jj-v275","Warn: Project is vulnerable to: GHSA-gxpj-cx7g-858c","Warn: Project is vulnerable to: GHSA-w573-4hg7-7wgq","Warn: Project is vulnerable to: GHSA-74fj-2j2h-c42q","Warn: Project is vulnerable to: GHSA-pw2r-vq6v-hr8c","Warn: Project is vulnerable to: GHSA-jchw-25xp-jwwc","Warn: Project is vulnerable to: GHSA-cxjh-pqwp-8mfp","Warn: Project is vulnerable to: GHSA-fjxv-7rqg-78g4","Warn: Project is vulnerable to: GHSA-43f8-2h32-f4cj","Warn: Project is vulnerable to: GHSA-896r-f27r-55mw","Warn: Project is vulnerable to: GHSA-9c47-m6qq-7p4h","Warn: Project is vulnerable to: GHSA-952p-6rrq-rcjv","Warn: Project is vulnerable to: GHSA-f8q6-p94x-37v3","Warn: Project is vulnerable to: GHSA-xvch-5gv4-984h","Warn: Project is vulnerable to: GHSA-rp65-9cf3-cjxr","Warn: Project is vulnerable to: GHSA-3j8f-xvm3-ffx4","Warn: Project is vulnerable to: GHSA-4p35-cfcx-8653","Warn: Project is vulnerable to: GHSA-7f3x-x4pr-wqhj","Warn: Project is vulnerable to: GHSA-jpp7-7chh-cf67","Warn: Project is vulnerable to: GHSA-q6wq-5p59-983w","Warn: Project is vulnerable to: GHSA-j9fq-vwqv-2fm2","Warn: Project is vulnerable to: GHSA-pqw5-jmp5-px4v","Warn: Project is vulnerable to: GHSA-hj48-42vr-x3v9","Warn: Project is vulnerable to: GHSA-hrpp-h998-j3pp","Warn: Project is vulnerable to: GHSA-9q5w-79cv-947m","Warn: Project is vulnerable to: GHSA-p8p7-x288-28g6","Warn: Project is vulnerable to: GHSA-c2qf-rxjj-qqgw","Warn: Project is vulnerable to: GHSA-4rq4-32rv-6wp6","Warn: Project is vulnerable to: GHSA-64g7-mvw6-v9qj","Warn: Project is vulnerable to: GHSA-jgrx-mgxx-jf9v","Warn: Project is vulnerable to: GHSA-72xf-g2v4-qvf3","Warn: Project is vulnerable to: GHSA-w5p7-h5w8-2hfq","Warn: Project is vulnerable to: GHSA-7p7h-4mm5-852v","Warn: Project is vulnerable to: GHSA-38fc-wpqx-33j7","Warn: Project is vulnerable to: GHSA-g3ch-rx76-35fx","Warn: Project is vulnerable to: GHSA-j8xg-fqg3-53r7","Warn: Project is vulnerable to: GHSA-6fc8-4gx4-v693","Warn: Project is vulnerable to: GHSA-3h5v-q93c-6h6q","Warn: Project is vulnerable to: GHSA-c4w7-xm78-47vh","Warn: Project is vulnerable to: GHSA-p9pc-299p-vxgp"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-21T13:21:47.932Z","repository_id":35010716,"created_at":"2025-08-21T13:21:47.932Z","updated_at":"2025-08-21T13:21:47.932Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279009263,"owners_count":26084570,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","election","javascript","library","module","nlp","npm","package","parse","politics","scrape","sentiment"],"created_at":"2024-11-21T02:09:33.380Z","updated_at":"2025-10-11T23:03:52.698Z","avatar_url":"https://github.com/mitevpi.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Poli Parse\n\n[![Generic badge](https://img.shields.io/badge/Docs-Web-Green.svg)](https://mitevpi.github.io/poli-parse/) [![Generic badge](https://img.shields.io/badge/Docs-MD-Green.svg)](docs/README.md) [![Generic badge](https://img.shields.io/badge/Samples-JS-Green.svg)](samples/) ![NPM](https://img.shields.io/npm/l/poli-parse.svg)\n\n[![npm](https://img.shields.io/npm/v/poli-parse.svg)](https://www.npmjs.com/package/poli-parse) [![npm bundle size](https://img.shields.io/bundlephobia/min/poli-parse.svg)](https://bundlephobia.com/result?/poli-parse) [![npm](https://img.shields.io/npm/dw/poli-parse.svg)](https://www.npmjs.com/package/poli-parse) [![npm2](https://img.shields.io/npm/dt/poli-parse.svg)](https://www.npmjs.com/package/poli-parse)\n\n[![GitHub issues](https://img.shields.io/github/issues/mitevpi/poli-parse.svg)](https://github.com/mitevpi/poli-parse/issues) ![David](https://img.shields.io/david/dev/mitevpi/poli-parse.svg) [![GitHub last commit](https://img.shields.io/github/last-commit/mitevpi/poli-parse.svg)](https://github.com/mitevpi/poli-parse/commits/master)\n\nPolitical news scraping \u0026 NLP parsing from web pages.\n\n## Usage\n\nTo use this module, install from [npm](https://www.npmjs.com/package/poli-parse), install locally using the command below, or clone this repository and import the .js files directly from source.\n\n```cmd\nnpm i poli-parse\n```\n\n### Imports\n\nImports can be done through the aggregating index.js file or via individual members.\n\n#### Full Import\n\n```js\nconst PoliParse = require('./index.js'); // from source\nconst PoliParse = require('poli-parse') // from npm\n\n// es6\nimport * as PoliParse from \"../src\"; // from source\nimport * as PoliParse from from \"poli-parse\"; // from npm\n```\n\n#### Individual Import\n\n```js\n// from source\nconst Parse = require(\"./Parse\");\n\nconst { Parse } = require(\"poli-parse\");\nimport { Parse } from \"poli-parse\"; // es6\n```\n\n### Methods\n\nThe collection below is just a sample of methods and may be out of date. For the most recent examples, please see the [samples](samples/) folder in the root directory of this repository.\n\n#### Basic\n\nBasic scrape/parse example.\n\n```js\nconst PP = require(\"../dist/index\"); // import library\n\n// hit the homepage of wsj.com\nPP.Scrape.AllText(\"https://www.wsj.com/\").then(data =\u003e {\n  // filter out text that isn't longer than two words (likely not a sentence)\n  const filtered = PP.Parse.FilterLength(data, 2);\n\n  // split monolithic text from the html into more useful chunks\n  const split = PP.Parse.SplitMonolithic(filtered);\n\n  // find all texts which mention a specific subject\n  const subject = PP.Parse.FilterSubject(split, [\n    \"Elizabeth\",\n    \"Warren\"\n  ]);\n\n  // see what headlines were found regarding the subject\n  console.log(subject);\n});\n```\n\n#### Advanced\n\nMore advanced scrape/parse example, with analysis logic added.\n\n```js\nconst PP = require(\"../dist/index\"); // import library\n\nPP.Scrape.AllText(\"https://www.wsj.com/\").then(data =\u003e {\n  let newData = PP.Parse.FilterLength(data, 2);\n  newData = PP.Parse.SplitMonolithic(newData);\n  newData = PP.Parse.FilterSubject(newData, [\"Donald\", \"Trump\"]);\n\n  const results = newData.map(async headline =\u003e {\n    const sentiment = await PP.Sentiment.Compute(headline);\n    const pos = await PP.Language.ComputePOS(headline);\n    return {\n      headline,\n      sentiment,\n      pos\n    };\n  });\n  Promise.all(results).then(completed =\u003e {\n    console.log(completed);\n  });\n});\n```\n\n## Commands\n\nThe following commands are available during development.\n\n```sh\nnpm test # run tests with Jest\nnpm run coverage # run tests with coverage and open it on browser\nnpm run lint # lint code\nnpm run docs # generate docs\nnpm run build # transpile code\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmitevpi%2Fpoli-parse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmitevpi%2Fpoli-parse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmitevpi%2Fpoli-parse/lists"}