{"id":20665387,"url":"https://github.com/rohit1901/substack-feed-api","last_synced_at":"2026-01-30T15:02:29.198Z","repository":{"id":245297298,"uuid":"817814809","full_name":"rohit1901/substack-feed-api","owner":"rohit1901","description":"A RSS Feed parser for Substack. #library","archived":false,"fork":false,"pushed_at":"2025-05-19T16:05:27.000Z","size":401,"stargazers_count":6,"open_issues_count":2,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-16T02:42:50.727Z","etag":null,"topics":["api","nodejs","substack","typescript"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/substack-feed-api","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rohit1901.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-06-20T13:50:26.000Z","updated_at":"2025-06-07T18:50:26.000Z","dependencies_parsed_at":"2024-06-21T07:48:25.865Z","dependency_job_id":"ddd38b16-50ae-4e5f-8c2e-7b2ce8bcacce","html_url":"https://github.com/rohit1901/substack-feed-api","commit_stats":null,"previous_names":["rohit1901/substack-feed-api"],"tags_count":20,"template":false,"template_full_name":null,"purl":"pkg:github/rohit1901/substack-feed-api","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rohit1901%2Fsubstack-feed-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rohit1901%2Fsubstack-feed-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rohit1901%2Fsubstack-feed-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rohit1901%2Fsubstack-feed-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rohit1901","download_url":"https://codeload.github.com/rohit1901/substack-feed-api/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rohit1901%2Fsubstack-feed-api/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279001643,"owners_count":26083146,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","nodejs","substack","typescript"],"created_at":"2024-11-16T19:30:38.300Z","updated_at":"2026-01-30T15:02:29.184Z","avatar_url":"https://github.com/rohit1901.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Substack Feed API\n\n`substack-feed-api` is a small TypeScript utility for turning RSS XML into typed objects using Cheerio, with first-class support for Substack and Goodreads feeds.\n\n## Features\n\n- **Type-safe** mapping from RSS XML to your own TypeScript types via generic selector maps.\n- Built-in helpers for Substack posts and Goodreads bookshelf RSS feeds (including shelves / reading status).\n- Uses Cheerio in XML mode, works well with namespaced tags like `content:encoded`.\n- Graceful error handling with configurable fallbacks and silent logging.\n\n## Installation\n\n```bash\nnpm install substack-feed-api\n# or\nyarn add substack-feed-api\n# or\npnpm add substack-feed-api\n```\n\n## Quick Start\n\n### Parsing Substack RSS\n\nSubstack exposes a standard RSS 2.0 feed with a `\u003cchannel\u003e` and multiple `\u003citem\u003e` entries; each item contains fields like `\u003ctitle\u003e`, `\u003cdescription\u003e`, `\u003clink\u003e`, `\u003cpubDate\u003e`, and `\u003ccontent:encoded\u003e` for the HTML body.\n\n```ts\nimport { parseSubstackRss, SubstackItem } from 'substack-feed-api';\n\nconst xml = await fetch('https://example.substack.com/feed').then(r =\u003e r.text());\n\nconst posts: SubstackItem[] = parseSubstackRss(xml);\n\n// Example item\n// {\n//   title: 'Both Not Half by Jassa Ahluwalia',\n//   description: 'A Humorous Journey Through Identity, Yet Lacking Cohesion',\n//   link: 'https://…',\n//   pubDate: 'Sun, 06 Oct 2024 15:35:17 GMT',\n//   content: '\u003cp\u003eJassa Ahluwalia\\'s \u003cstrong\u003eBoth Not Half\u003c/strong\u003e…'\n// }\n```\n\nYou can override any selector if your feed schema differs:\n\n```ts\nconst postsCustom = parseSubstackRss(xml, {\n  selectors: {\n    // use \u003cdescription\u003e as content\n    content: 'description',\n  },\n});\n```\n\n### Parsing Goodreads Bookshelf RSS\n\nGoodreads’ “bookshelf” RSS feed exposes many book-related tags per `\u003citem\u003e` (e.g. `\u003ctitle\u003e`, `\u003cbook_description\u003e`, `\u003cbook_large_image_url\u003e`, `\u003cauthor_name\u003e`, `\u003cuser_shelves\u003e`).\n\nThe library exposes a Goodreads-specific helper that returns a higher-level `GoodreadsReadingState`:\n\n```ts\nimport {\n  parseGoodreadsRss,\n  GoodreadsReadingState,\n} from 'substack-feed-api';\n\nconst xml = await fetch('\u003cgoodreads-list-rss-url\u003e').then(r =\u003e r.text());\n\nconst states: GoodreadsReadingState[] = parseGoodreadsRss(xml);\n\n// Example shape:\n// {\n//   status: 'WANTS_TO_READ' | 'IS_READING' | 'FINISHED',\n//   book: {\n//     title: 'Malice (Detective Kaga, #1)',\n//     description: 'Acclaimed bestselling novelist Kunihiko Hidaka is found brutally murdered…',\n//     cover: 'https://i.gr-assets.com/.../20613611._SY475_.jpg',\n//     authors: [{ name: 'Keigo Higashino' }]\n//   }\n// }\n```\n\nBy default, the Goodreads parser derives status from `user_shelves` (e.g. `to-read`, `currently-reading`, `read`).\n\nYou can still adjust selectors if Goodreads ever changes tag names:\n\n```ts\nconst customStates = parseGoodreadsRss(xml, {\n  selectors: {\n    // Example: use medium image instead of large\n    cover: 'book_medium_image_url',\n  },\n});\n```\n\n## API\n\n### `parseRssItems` – Generic Core\n\n```ts\nfunction parseRssItems\u003cTRaw extends Record\u003cstring, string\u003e\u003e(\n  xml: string,\n  options?: {\n    itemSelector?: string;\n    selectors?: Partial\u003cRecord\u003ckeyof TRaw, string\u003e\u003e;\n    fallback?: TRaw[];\n  }\n): TRaw[];\n```\n\n- `xml`: Full RSS XML string.\n- `itemSelector`: CSS selector for each RSS item node, default `'channel \u003e item'`.\n- `selectors`: Map from property name → CSS selector **relative to each item node**.\n- `fallback`: Array to return if parsing fails (e.g., malformed XML); error is logged to `console.error` but not thrown.\n\nExample: Minimal Generic Usage:\n\n```ts\ntype MinimalItem = {\n  title: string;\n  link: string;\n};\n\nconst items = parseRssItems\u003cMinimalItem\u003e(xml, {\n  selectors: {\n    title: 'title',\n    link: 'link',\n  },\n});\n```\n\n### `parseSubstackRss`\n\n```ts\ntype SubstackItem = {\n  title: string;\n  description: string;\n  link: string;\n  pubDate: string;\n  content: string;\n};\n\nfunction parseSubstackRss(\n  xml: string,\n  options?: {\n    itemSelector?: string;\n    selectors?: Partial\u003cRecord\u003ckeyof SubstackItem, string\u003e\u003e;\n    fallback?: SubstackItem[];\n  }\n): SubstackItem[];\n```\n\nDefault selectors (overridable):\n\n```ts\n{\n  title: 'title',\n  description: 'description',\n  link: 'link',\n  pubDate: 'pubDate',\n  content: 'content\\\\:encoded',\n}\n```\n\nThis matches typical Substack feeds which use `content:encoded` for the full HTML article body.\n\n### `parseGoodreadsRss`\n\n```ts\ntype BookAuthor = { name: string };\n\ntype GoodreadsBook = {\n  title: string;\n  description: string;\n  cover: string;\n  authors?: BookAuthor[];\n};\n\ntype GoodreadsReadingStatus = 'IS_READING' | 'FINISHED' | 'WANTS_TO_READ';\n\ntype GoodreadsReadingState = {\n  book: GoodreadsBook;\n  status: GoodreadsReadingStatus;\n};\n\nfunction parseGoodreadsRss(\n  xml: string,\n  options?: {\n    itemSelector?: string;\n    selectors?: Partial\u003c{\n      title: string;\n      description: string;\n      cover: string;\n      author: string;\n      shelves: string;\n    }\u003e;\n    fallback?: GoodreadsReadingState[]; // via raw fallback mapping\n  }\n): GoodreadsReadingState[];\n```\n\nDefault Goodreads selectors map RSS tags to an internal flat type:\n\n```ts\n{\n  title: 'title',\n  description: 'book_description',\n  cover: 'book_large_image_url',\n  author: 'author_name',\n  shelves: 'user_shelves',\n}\n```\n\nThe parser then:\n\n- Builds a flat raw record from each `\u003citem\u003e`.\n- Maps `shelves` to a `GoodreadsReadingStatus` (e.g., `currently-reading` → `IS_READING`, `read` → `FINISHED`, otherwise `WANTS_TO_READ`).\n- Wraps book information into `GoodreadsBook` and `BookAuthor`.\n\n## Error Handling\n\nAll parsing functions follow the same pattern:\n\n- Wrap parsing and traversal in a `try/catch`.\n- On error, log a concise entry to `console.error` with context (selectors, item selector).\n- Return the provided `fallback` (default `[]`) instead of throwing.\n\nExample:\n\n```ts\nconst items = parseSubstackRss('\u003cinvalid-xml\u003e', {\n  fallback: [],\n}); // returns [], logs an error, does not crash your app\n```\n\nThis makes the library safe to use in background jobs, CLI tools, or edge handlers where a single bad feed should not bring down the entire process.\n\n## Extending for Other Feeds\n\nTo support another RSS feed type, you generally:\n\n1. Define a flat `TRaw` type that contains only string fields.\n2. Call `parseRssItems\u003cTRaw\u003e` with a selector map that matches the feed’s tags.\n3. Map `TRaw` to your domain model in a small wrapper, similar to `parseGoodreadsRss`.\n\nExample skeleton:\n\n```ts\ntype MyFeedRaw = {\n  title: string;\n  summary: string;\n  link: string;\n};\n\ntype MyFeedItem = {\n  title: string;\n  summary: string;\n  url: string;\n};\n\nfunction parseMyFeed(xml: string): MyFeedItem[] {\n  const raw = parseRssItems\u003cMyFeedRaw\u003e(xml, {\n    selectors: {\n      title: 'title',\n      summary: 'summary',\n      link: 'link',\n    },\n  });\n\n  return raw.map(r =\u003e ({\n    title: r.title,\n    summary: r.summary,\n    url: r.link,\n  }));\n}\n```\n\n## License\n\nThis project is licensed under the MIT License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frohit1901%2Fsubstack-feed-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frohit1901%2Fsubstack-feed-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frohit1901%2Fsubstack-feed-api/lists"}