{"id":13583422,"url":"https://github.com/croqaz/clean-mark","last_synced_at":"2025-10-11T04:37:32.131Z","repository":{"id":48239569,"uuid":"90354129","full_name":"croqaz/clean-mark","owner":"croqaz","description":"Convert an article into clean text","archived":false,"fork":false,"pushed_at":"2022-09-12T10:32:49.000Z","size":1083,"stargazers_count":603,"open_issues_count":3,"forks_count":51,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-03-28T03:12:36.894Z","etag":null,"topics":["article","markdown","readability","text"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/croqaz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"docs/CODE-OF-CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-05-05T08:24:37.000Z","updated_at":"2025-03-06T14:04:28.000Z","dependencies_parsed_at":"2022-08-12T19:41:26.021Z","dependency_job_id":null,"html_url":"https://github.com/croqaz/clean-mark","commit_stats":null,"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/croqaz%2Fclean-mark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/croqaz%2Fclean-mark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/croqaz%2Fclean-mark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/croqaz%2Fclean-mark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/croqaz","download_url":"https://codeload.github.com/croqaz/clean-mark/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247117765,"owners_count":20886439,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["article","markdown","readability","text"],"created_at":"2024-08-01T15:03:28.166Z","updated_at":"2025-10-11T04:37:27.079Z","avatar_url":"https://github.com/croqaz.png","language":"JavaScript","readme":"# ➹ Clean-mark\n\n  \u003e Convert a blog article into a clean Markdown text file.\n\n  [![NPM Version][npm-image]][npm-url]\n  [![NPM Downloads][downloads-image]][downloads-url]\n  [![Build Status][build-image]][build-url]\n  [![Standard Style Guide][style-image]][style-url]\n\n\n## Example\n\nFor example, this article:\n\n[![Original article](article-screen.png)](http://www.techradar.com/news/amd-vs-intel-the-16-core-showdown)\n\nIs converted into this text file:\n\n![Clean text](clean-screen.png)\n\n\n## Usage\n\n\u003e $ clean-mark \"http://some-website.com/fancy-article\"\n\nThe article will be automatically named using the URL path name. In the case, above, the name will be `fancy-article.md`.\n\nThe file type can be specified:\n\n\u003e $ clean-mark \"http://some-website.com/fancy-article\" -t html\n\nThe available types are: HTML, TEXT and Markdown.\n\nThe output file and path can be also specified:\n\n\u003e $ clean-mark \"http://some-website.com/fancy-article\" -o /tmp/article\n\nIn that case the output will be `/tmp/article.md`. The extension is added automatically.\n\n\n## Installation\n\nSimply install with npm:\n\n\u003e $ npm install clean-mark --global\n\n\n## Why ?\n\n* to save interesting articles offline, in a highly readable text format\n* it's easy to read on a tablet, or a Kindle (as it is, or exported to PDF)\n* Markdown is easy to export into different formats\n* for offline text analysis of multiple articles, using machine learning / AI\n\n\n## How ?\n\nImplementation steps:\n\n1. Downloads the content of a web page\n1. Meta-scrape page details (title, author, date, etc)\n1. Sanitizes the ugly HTML\n1. Minifies the disinfected HTML\n1. Converts the result into clean Markdown text\n\n\nThis project depends on the [A-Extractor project](https://github.com/croqaz/a-extractor), a database of expressions used for extracting content from blogs and articles.\n\n\n## Vision\n\nThe goals of the project are are:\n\n1. Good text extraction\n1. More useless text is preferred, instead of cutting from the actual article by mistake\n1. Extracting media (images, videos, audio) is not that important\n1. Extraction speed is not that important\n\n\n## Contributing\n\nClean-mark was tested on all major news sites. On some websites, the text, or links are cut from the article.\nIn this case, you have to manually edit the resulted text,\n\nAND\n\nplease raise an [issue on A-Extractor](https://github.com/croqaz/a-extractor/issues) with the link that doesn't work and we'll add it in the database, so that next time, the text will be extracted correctly.\n\nAlso, see [how to contribute](CONTRIBUTING.md).\n\n\n## Other readability tools\n\n* http://ejucovy.github.io/readability :: Original readability ☠\n* https://github.com/luin/readability :: Next readability ☠\n* https://github.com/olragon/node-readability :: Node.js readability ☠\n* https://github.com/bndr/node-read : Node read ☠\n* https://github.com/mozilla/readability :: Mozilla's readability, up to date\n* https://github.com/codelucas/newspaper :: Lucas Ou-Yang's Newspaper3k\n* https://github.com/postlight/mercury-parser :: Postlight's Mercury Parser\n* https://github.com/ageitgey/node-unfluff :: Automatically extract body content (and other cool stuff) from HTML document\n\n## Web archiving tools\n\n* https://github.com/croqaz/web-snap :: create \"perfect\" snapshots of web pages\n* https://github.com/Y2Z/monolith :: Rust CLI app\n* https://github.com/go-shiori/obelisk :: Go CLI app, inspired by Monolith\n* https://github.com/wabarc/cairn :: TypeScript implementation of Obelisk\n* https://github.com/gildas-lormeau/SingleFile :: Web Extension for Firefox/Chrome/Edge and CLI tool to save a web page in a single HTML file\n\nMore archiving tools:\n\n* https://crlf.link/mem/web-archiving\n* https://github.com/iipc/awesome-web-archiving\n\n-----\n\n## License\n\n[MIT](LICENSE) © Cristi Constantin.\n\n\n[npm-image]: https://img.shields.io/npm/v/clean-mark.svg\n[npm-url]: https://www.npmjs.com/package/clean-mark\n[david-image]: https://david-dm.org/croqaz/clean-mark/status.svg\n[david-url]: https://david-dm.org/croqaz/clean-mark\n[downloads-image]: https://img.shields.io/npm/dm/clean-mark.svg\n[downloads-url]: https://npmjs.org/package/clean-mark\n[build-image]: https://travis-ci.org/croqaz/clean-mark.svg?branch=master\n[build-url]: https://travis-ci.org/croqaz/clean-mark\n[style-image]: https://img.shields.io/badge/code_style-standard-brightgreen.svg\n[style-url]: https://standardjs.com\n","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcroqaz%2Fclean-mark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcroqaz%2Fclean-mark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcroqaz%2Fclean-mark/lists"}