{"id":14865239,"url":"https://github.com/timf34/Substack2Markdown","last_synced_at":"2025-09-18T20:32:26.541Z","repository":{"id":177942439,"uuid":"660398700","full_name":"timf34/Substack2Markdown","owner":"timf34","description":"Download/ export free and premium Substack posts, saving them as Markdown files. Also generates HTML interfaces to allow you to browse and sort the markdown files for each author.","archived":false,"fork":false,"pushed_at":"2024-08-22T01:27:08.000Z","size":432,"stargazers_count":164,"open_issues_count":4,"forks_count":32,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-05T17:07:38.206Z","etag":null,"topics":["html","javascript","markdown","python","scraper","selenium","substack","substack-download","substack-scraper-python","ui"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timf34.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-29T23:32:00.000Z","updated_at":"2025-01-04T16:01:51.000Z","dependencies_parsed_at":"2024-01-06T03:22:14.418Z","dependency_job_id":"8d932413-1f3d-4ff7-b545-e9e83c6eb4f9","html_url":"https://github.com/timf34/Substack2Markdown","commit_stats":null,"previous_names":["timf34/premiumsubstackscraper","timf34/substackscraper","timf34/substack2markdown"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timf34%2FSubstack2Markdown","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timf34%2FSubstack2Markdown/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timf34%2FSubstack2Markdown/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timf34%2FSubstack2Markdown/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timf34","download_url":"https://codeload.github.com/timf34/Substack2Markdown/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233520004,"owners_count":18688530,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["html","javascript","markdown","python","scraper","selenium","substack","substack-download","substack-scraper-python","ui"],"created_at":"2024-09-20T00:02:10.779Z","updated_at":"2025-09-18T20:32:26.532Z","avatar_url":"https://github.com/timf34.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Substack2Markdown\r\n\r\nSubstack2Markdown is a Python tool for downloading free and premium Substack posts and saving them as both Markdown and \r\nHTML files, and includes a simple HTML interface to browse and sort through the posts. It will save paid for content as \r\nlong as you're subscribed to that substack. \r\n\r\n🆕 @Firevvork has built a web version of this tool at [Substack Reader](https://www.substacktools.com/reader) - no \r\ninstallation required! (Works for free Substacks only.)\r\n\r\n\r\n![Substack2Markdown Interface](./assets/images/screenshot.png)\r\n\r\nOnce you run the script, it will create a folder named after the substack in `/substack_md_files`,\r\nand then begin to scrape the substack URL, converting the blog posts into markdown files. Once all the posts have been\r\nsaved, it will generate an HTML file in `/substack_html_pages` directory that allows you to browse the posts.\r\n\r\nYou can either hardcode the substack URL and the number of posts you'd like to save into the top of the file, or \r\nspecify them as command line arguments.\r\n\r\n## Features\r\n\r\n- Converts Substack posts into Markdown files.\r\n- Generates an HTML file to browse Markdown files.\r\n- Supports free and premium content (with subscription).\r\n- The HTML interface allows sorting essays by date or likes.\r\n\r\n## Installation\r\n\r\nClone the repo and install the dependencies:\r\n\r\n```bash\r\ngit clone https://github.com/yourusername/substack_scraper.git\r\ncd substack_scraper\r\n\r\n# # Optinally create a virtual environment\r\n# python -m venv venv\r\n# # Activate the virtual environment\r\n# .\\venv\\Scripts\\activate  # Windows\r\n# source venv/bin/activate  # Linux\r\n\r\npip install -r requirements.txt\r\n```\r\n\r\nFor the premium scraper, update the `config.py` in the root directory with your Substack email and password:\r\n\r\n```python\r\nEMAIL = \"your-email@domain.com\"\r\nPASSWORD = \"your-password\"\r\n```\r\n\r\nYou'll also need Microsoft Edge installed for the Selenium webdriver.\r\n\r\n## Usage\r\n\r\nSpecify the Substack URL and the directory to save the posts to:\r\n\r\nYou can hardcode your desired Substack URL and the number of posts you'd like to save into the top of the file and run:\r\n```bash\r\npython substack_scraper.py\r\n```\r\n\r\nFor free Substack sites:\r\n\r\n```bash\r\npython substack_scraper.py --url https://example.substack.com --directory /path/to/save/posts\r\n```\r\n\r\nFor premium Substack sites:\r\n\r\n```bash\r\npython substack_scraper.py --url https://example.substack.com --directory /path/to/save/posts --premium\r\n```\r\n\r\nTo scrape a specific number of posts:\r\n\r\n```bash\r\npython substack_scraper.py --url https://example.substack.com --directory /path/to/save/posts --number 5\r\n```\r\n\r\n### Online Version\r\n\r\nFor a hassle-free experience without any local setup:\r\n\r\n1. Visit [Substack Reader](https://www.substacktools.com/reader)\r\n2. Enter the Substack URL you want to read or export\r\n3. Click \"Go\" to instantly view the content or \"Export\" to download Markdown files\r\n\r\nThis online version provides a user-friendly web interface for reading and exporting free Substack articles, with no installation required. However, please note that the online version currently does not support exporting premium content. For full functionality, including premium content export, please use the local script as described above. Built by @Firevvork. \r\n\r\n## Viewing Markdown Files in Browser\r\n\r\nTo read the Markdown files in your browser, install the [Markdown Viewer](https://chromewebstore.google.com/detail/markdown-viewer/ckkdlimhmcjmikdlpkmbgfkaikojcbjk)\r\nbrowser extension. But note, we also save the files as HTML for easy viewing, \r\njust set the toggle to HTML on the author homepage. \r\n\r\nOr you can use our [Substack Reader](https://www.substacktools.com/reader) online tool, which allows you to read and export free Substack articles directly in your browser. (Note: Premium content export is currently only available in the local script version)\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimf34%2FSubstack2Markdown","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimf34%2FSubstack2Markdown","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimf34%2FSubstack2Markdown/lists"}