{"id":51362725,"url":"https://github.com/profullstack/summary-forge-module","last_synced_at":"2026-07-03T00:31:40.536Z","repository":{"id":322726009,"uuid":"1090652435","full_name":"profullstack/summary-forge-module","owner":"profullstack","description":"An intelligent tool that uses OpenAI's GPT-5 to forge comprehensive summaries of ebooks in multiple formats.","archived":false,"fork":false,"pushed_at":"2025-11-26T12:50:42.000Z","size":824,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-11-29T10:54:02.591Z","etag":null,"topics":["amazon","ebooks","elevenlabs","epub","openai","pdf","rainforestapi","summary"],"latest_commit_sha":null,"homepage":"https://summaryforge.com","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/profullstack.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-06T00:38:22.000Z","updated_at":"2025-11-26T12:50:46.000Z","dependencies_parsed_at":"2025-11-06T03:07:32.906Z","dependency_job_id":null,"html_url":"https://github.com/profullstack/summary-forge-module","commit_stats":null,"previous_names":["profullstack/summary-forge-module"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/profullstack/summary-forge-module","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/profullstack%2Fsummary-forge-module","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/profullstack%2Fsummary-forge-module/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/profullstack%2Fsummary-forge-module/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/profullstack%2Fsummary-forge-module/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/profullstack","download_url":"https://codeload.github.com/profullstack/summary-forge-module/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/profullstack%2Fsummary-forge-module/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35068133,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-02T02:00:06.368Z","response_time":173,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon","ebooks","elevenlabs","epub","openai","pdf","rainforestapi","summary"],"created_at":"2026-07-03T00:31:39.647Z","updated_at":"2026-07-03T00:31:40.521Z","avatar_url":"https://github.com/profullstack.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Summary Forge Module\n\nAn intelligent tool that uses OpenAI's GPT-5 to forge comprehensive summaries of ebooks in multiple formats.\n\n**Repository:** [git@github.com:profullstack/summary-forge-module.git](https://github.com/profullstack/summary-forge-module)\n\n## Features\n\n- 📚 **Multiple Input Formats**: Supports PDF, EPUB files, and web page URLs\n- 🌐 **Web Page Summarization**: Fetch and summarize any web page with automatic content extraction\n- 🤖 **AI-Powered Summaries**: Uses GPT-5 with direct PDF upload for better quality\n- 📊 **Vision API**: Preserves formatting, tables, diagrams, and images from PDFs\n- 🧩 **Intelligent Chunking**: Automatically processes large PDFs (500+ pages) without truncation\n- 🛡️ **Directory Protection**: Prompts before overwriting existing summaries (use --force to skip)\n- 📦 **Multiple Output Formats**: Creates Markdown, PDF, EPUB, plain text, and MP3 audio summaries\n- 🃏 **Printable Flashcards**: Generates double-sided flashcard PDFs for studying\n- 🖼️ **Flashcard Images**: Individual PNG images for web app integration (q-001.png, a-001.png, etc.)\n- 🎙️ **Natural Audio Narration**: AI-generated conversational audio script for better listening\n- 🗜️ **Bundled Output**: Packages everything into a convenient `.tgz` archive\n- 🔄 **Auto-Conversion**: Automatically converts EPUB to PDF using Calibre\n- 🔍 **Book Search**: Search Amazon by title using Rainforest API\n- 📖 **Auto-Download**: Downloads books from Anna's Archive with CAPTCHA solving\n- 💻 **CLI \u0026 Module**: Use as a command-line tool or import as an ESM module\n- 🎨 **Interactive Mode**: Guided workflow with inquirer prompts\n- 📥 **EPUB Priority**: Automatically prefers EPUB format (open standard, more flexible)\n\n## Installation\n\n### Global Installation (CLI)\n\n```bash\npnpm install -g @profullstack/summary-forge-module\n```\n\n### Local Installation (Module)\n\n```bash\npnpm add @profullstack/summary-forge-module\n```\n\n## Prerequisites\n\n1. **Node.js** v20 or newer\n\n2. **Calibre** (for EPUB conversion - provides `ebook-convert` command)\n   ```bash\n   # macOS\n   brew install calibre\n   \n   # Ubuntu/Debian\n   sudo apt-get install calibre\n   \n   # Arch Linux\n   sudo pacman -S calibre\n   ```\n\n3. **Pandoc** (for document conversion)\n   ```bash\n   # macOS\n   brew install pandoc\n   \n   # Ubuntu/Debian\n   sudo apt-get install pandoc\n   \n   # Arch Linux\n   sudo pacman -S pandoc\n   ```\n\n4. **XeLaTeX** (for PDF generation)\n   ```bash\n   # macOS\n   brew install --cask mactex\n   \n   # Ubuntu/Debian\n   sudo apt-get install texlive-xetex\n   \n   # Arch Linux\n   sudo pacman -S texlive-core texlive-xetex\n   ```\n\n## CLI Usage\n\n### First-Time Setup\n\nBefore using the CLI, configure your API keys:\n\n```bash\nsummary setup\n```\n\nThis interactive command will prompt you for:\n- **OpenAI API Key** (required)\n- **Rainforest API Key** (optional - for Amazon book search)\n- **ElevenLabs API Key** (optional - for audio generation, [get key here](https://try.elevenlabs.io/oh7kgotrpjnv))\n- **2Captcha API Key** (optional - for CAPTCHA solving, [sign up here](https://2captcha.com/?from=9630996))\n- **Browserless API Key** (optional)\n- Browser and proxy settings\n\nConfiguration is saved to `~/.config/summary-forge/settings.json` and used automatically by all CLI commands.\n\n### Managing Configuration\n\n```bash\n# View current configuration\nsummary config\n\n# Update configuration\nsummary setup\n\n# Delete configuration\nsummary config --delete\n```\n\n**Note:** The CLI will use configuration in this priority order:\n1. Environment variables (`.env` file)\n2. Configuration file (`~/.config/summary-forge/settings.json`)\n\n### Interactive Mode (Recommended)\n\n```bash\nsummary interactive\n# or\nsummary i\n```\n\nThis launches an interactive menu where you can:\n- Process local files (PDF/EPUB)\n- Process web page URLs\n- Search for books by title\n- Look up books by ISBN/ASIN\n\n### Process a File\n\n```bash\nsummary file /path/to/book.pdf\nsummary file /path/to/book.epub\n\n# Force overwrite if directory already exists\nsummary file /path/to/book.pdf --force\nsummary file /path/to/book.pdf -f\n```\n\n### Process a Web Page URL\n\n```bash\nsummary url https://example.com/article\nsummary url https://blog.example.com/post/123\n\n# Force overwrite if directory already exists\nsummary url https://example.com/article --force\nsummary url https://example.com/article -f\n```\n\n**Features:**\n- Automatically fetches web page content using Puppeteer\n- Sanitizes HTML to remove navigation, ads, footers, and other non-content elements\n- Saves web page as PDF for processing\n- Generates clean title from page title or uses OpenAI to create one\n- Prompts specifically optimized for web page content (ignores nav/ads/footers)\n- Creates same output formats as book processing (MD, TXT, PDF, EPUB, MP3, flashcards)\n\n\n### Search by Title\n\n```bash\n# Search for books (defaults to 1lib.sk - faster, no DDoS protection)\nsummary search \"LLM Fine Tuning\"\nsummary search \"JavaScript\" --max-results 5 --extensions pdf,epub\nsummary search \"Python\" --year-from 2020 --year-to 2024\nsummary search \"Machine Learning\" --languages english --order date\n\n# Use Anna's Archive instead (has DDoS protection, slower)\nsummary search \"Clean Code\" --source anna\nsummary search \"Rare Book\" --source anna --sources zlib,lgli\n\n# Title search (shortcut for search command)\nsummary title \"A Philosophy of Software Design\"\nsummary title \"Clean Code\" --force  # Auto-select first result\nsummary title \"Python\" --source anna  # Use Anna's Archive\n\n# ISBN lookup (defaults to 1lib.sk)\nsummary isbn 9780134685991\nsummary isbn B075HYVHWK --force  # Auto-select and process\nsummary isbn 9780134685991 --source anna  # Use Anna's Archive\n\n# Common Options:\n#   --source \u003csource\u003e              Search source: zlib (1lib.sk, default) or anna (Anna's Archive)\n#   -n, --max-results \u003cnumber\u003e     Maximum results to display (default: 10)\n#   -f, --force                    Auto-select first result and process immediately\n#\n# 1lib.sk Options (--source zlib, default):\n#   --year-from \u003cyear\u003e             Filter by publication year from (e.g., 2020)\n#   --year-to \u003cyear\u003e               Filter by publication year to (e.g., 2024)\n#   -l, --languages \u003clanguages\u003e    Language filter, comma-separated (default: english)\n#   -e, --extensions \u003cextensions\u003e  File extensions, comma-separated (case-insensitive, default: PDF)\n#   --content-types \u003ctypes\u003e        Content types, comma-separated (default: book)\n#   -s, --order \u003corder\u003e            Sort order: date (newest) or empty for relevance\n#   --view \u003cview\u003e                  View type: list or grid (default: list)\n#\n# Anna's Archive Options (--source anna):\n#   -f, --format \u003cformat\u003e          Filter by format: pdf, epub, pdf,epub, or all (default: pdf)\n#   -s, --sort \u003csort\u003e              Sort by: date (newest) or empty for relevance (default: '')\n#   -l, --language \u003clanguage\u003e      Language code(s), comma-separated (e.g., en, es, fr) (default: en)\n#   --sources \u003csources\u003e            Data sources, comma-separated (default: all sources)\n#                                  Options: zlib, lgli, lgrs, and others\n```\n\n### Look up by ISBN/ASIN\n\n```bash\nsummary isbn B075HYVHWK\n\n# Force overwrite if directory already exists\nsummary isbn B075HYVHWK --force\nsummary isbn B075HYVHWK -f\n```\n\n### Help\n\n```bash\nsummary --help\nsummary file --help\n```\n\n## Programmatic Usage\n\n### JSON API Format\n\n**All methods now return consistent JSON objects** with the following structure:\n\n```javascript\n{\n  success: true | false,  // Indicates if operation succeeded\n  ...data,                // Method-specific data fields\n  error?: string,         // Error message (only when success is false)\n  message?: string        // Success message (optional)\n}\n```\n\nThis enables:\n- ✅ **Consistent error handling** - Check `success` field instead of try-catch\n- ✅ **REST API ready** - Direct JSON responses for HTTP endpoints\n- ✅ **Better debugging** - Rich metadata in all responses\n- ✅ **Type-safe** - Predictable structure for TypeScript users\n\n### Basic Example\n\n```javascript\nimport { SummaryForge } from '@profullstack/summary-forge-module';\nimport { loadConfig } from '@profullstack/summary-forge-module/config';\n\n// Load config from ~/.config/summary-forge/settings.json\nconst configResult = await loadConfig();\nif (!configResult.success) {\n  console.error('Failed to load config:', configResult.error);\n  process.exit(1);\n}\n\nconst forge = new SummaryForge(configResult.config);\n\nconst result = await forge.processFile('./my-book.pdf');\nif (result.success) {\n  console.log('Summary created:', result.archive);\n  console.log('Files:', result.files);\n  console.log('Costs:', result.costs);\n} else {\n  console.error('Processing failed:', result.error);\n}\n```\n\n### Configuration Options\n\n```javascript\nimport { SummaryForge } from '@profullstack/summary-forge-module';\n\nconst forge = new SummaryForge({\n  // Required\n  openaiApiKey: 'sk-...',\n  \n  // Optional API keys\n  rainforestApiKey: 'your-key',      // For Amazon search\n  elevenlabsApiKey: 'sk-...',        // For audio generation (get key: https://try.elevenlabs.io/oh7kgotrpjnv)\n  twocaptchaApiKey: 'your-key',      // For CAPTCHA solving (sign up: https://2captcha.com/?from=9630996)\n  browserlessApiKey: 'your-key',     // For browserless.io\n  \n  // Processing options\n  maxChars: 500000,                  // Max chars to process\n  maxTokens: 20000,                  // Max tokens in output summary\n  maxInputTokens: 250000,            // Max input tokens per API call (default: 250000 for GPT-5)\n  \n  // Audio options\n  voiceId: '21m00Tcm4TlvDq8ikWAM',  // ElevenLabs voice\n  voiceSettings: {\n    stability: 0.5,\n    similarity_boost: 0.75\n  },\n  \n  // Browser options\n  headless: true,                    // Run browser in headless mode\n  enableProxy: false,                // Enable proxy\n  proxyUrl: 'http://proxy.com',     // Proxy URL\n  proxyUsername: 'user',             // Proxy username\n  proxyPassword: 'pass',             // Proxy password\n  proxyPoolSize: 36                  // Number of proxies in pool (default: 36)\n});\n\nconst result = await forge.processFile('./book.epub');\nconsole.log('Archive:', result.archive);\n```\n\n### Search for Books\n\n#### Using Amazon/Rainforest API\n\n```javascript\nconst forge = new SummaryForge({\n  openaiApiKey: process.env.OPENAI_API_KEY,\n  rainforestApiKey: process.env.RAINFOREST_API_KEY\n});\n\nconst searchResult = await forge.searchBookByTitle('Clean Code');\nif (!searchResult.success) {\n  console.error('Search failed:', searchResult.error);\n  process.exit(1);\n}\n\nconsole.log(`Found ${searchResult.count} results:`);\nconsole.log(searchResult.results.map(b =\u003e ({\n  title: b.title,\n  author: b.author,\n  asin: b.asin\n})));\n\n// Get download URL\nconst url = forge.getAnnasArchiveUrl(searchResult.results[0].asin);\nconsole.log('Download from:', url);\n```\n\n#### Using Anna's Archive Direct Search (No Rainforest API Required)\n\n```javascript\nconst forge = new SummaryForge({\n  openaiApiKey: process.env.OPENAI_API_KEY,\n  enableProxy: true,\n  proxyUrl: process.env.PROXY_URL,\n  proxyUsername: process.env.PROXY_USERNAME,\n  proxyPassword: process.env.PROXY_PASSWORD\n});\n\n// Basic search\nconst searchResult = await forge.searchAnnasArchive('JavaScript', {\n  maxResults: 10,\n  format: 'pdf',\n  sortBy: 'date'  // Sort by newest\n});\n\nif (!searchResult.success) {\n  console.error('Search failed:', searchResult.error);\n  process.exit(1);\n}\n\nconsole.log(`Found ${searchResult.count} results`);\nconsole.log(searchResult.results.map(r =\u003e ({\n  title: r.title,\n  author: r.author,\n  format: r.format,\n  size: `${r.sizeInMB.toFixed(1)} MB`,\n  url: r.url\n})));\n\n// Download the first result\nif (searchResult.results.length \u003e 0) {\n  const md5 = searchResult.results[0].href.match(/\\/md5\\/([a-f0-9]+)/)[1];\n  const downloadResult = await forge.downloadFromAnnasArchive(md5, '.', searchResult.results[0].title);\n  \n  if (downloadResult.success) {\n    console.log('Downloaded:', downloadResult.filepath);\n    console.log('Directory:', downloadResult.directory);\n  } else {\n    console.error('Download failed:', downloadResult.error);\n  }\n}\n```\n\n#### Using 1lib.sk Search (Faster, No DDoS Protection)\n\n```javascript\nconst forge = new SummaryForge({\n  openaiApiKey: process.env.OPENAI_API_KEY,\n  enableProxy: true,\n  proxyUrl: process.env.PROXY_URL,\n  proxyUsername: process.env.PROXY_USERNAME,\n  proxyPassword: process.env.PROXY_PASSWORD\n});\n\n// Basic search\nconst searchResult = await forge.search1lib('LLM Fine Tuning', {\n  maxResults: 10,\n  yearFrom: 2020,\n  languages: ['english'],\n  extensions: ['PDF']\n});\n\nif (!searchResult.success) {\n  console.error('Search failed:', searchResult.error);\n  process.exit(1);\n}\n\nconsole.log(`Found ${searchResult.count} results`);\nconsole.log(searchResult.results.map(r =\u003e ({\n  title: r.title,\n  author: r.author,\n  year: r.year,\n  extension: r.extension,\n  size: r.size,\n  language: r.language,\n  isbn: r.isbn,\n  url: r.url\n})));\n\n// Download the first result\nif (searchResult.results.length \u003e 0) {\n  const downloadResult = await forge.downloadFrom1lib(\n    searchResult.results[0].url,\n    '.',\n    searchResult.results[0].title\n  );\n  \n  if (downloadResult.success) {\n    console.log('Downloaded:', downloadResult.filepath);\n    \n    // Process the downloaded book\n    const processResult = await forge.processFile(downloadResult.filepath, downloadResult.identifier);\n    if (processResult.success) {\n      console.log('Summary created:', processResult.archive);\n      console.log('Costs:', processResult.costs);\n    } else {\n      console.error('Processing failed:', processResult.error);\n    }\n  } else {\n    console.error('Download failed:', downloadResult.error);\n  }\n}\n```\n\n**Enhanced Error Handling:**\n\nThe 1lib.sk download functionality includes robust error handling with automatic debugging:\n\n- **Multiple Selector Fallbacks**: Tries 6 different selectors to find download buttons\n- **Debug HTML Capture**: Saves page HTML when download button isn't found\n- **Link Analysis**: Lists all links on the page for troubleshooting\n- **Detailed Error Messages**: Provides actionable information for debugging\n\nIf a download fails, check the `debug-book-page.html` file in the book's directory for detailed page structure information.\n\n### API Reference\n\n#### Constructor Options\n\n```javascript\nnew SummaryForge({\n  // API Keys\n  openaiApiKey: string,      // Required: OpenAI API key\n  rainforestApiKey: string,  // Optional: For title search\n  elevenlabsApiKey: string,  // Optional: For audio generation\n  twocaptchaApiKey: string,  // Optional: For CAPTCHA solving\n  browserlessApiKey: string, // Optional: For browserless.io\n  \n  // Processing Options\n  maxChars: number,          // Optional: Max chars to process (default: 400000)\n  maxTokens: number,         // Optional: Max tokens in output summary (default: 16000)\n  maxInputTokens: number,    // Optional: Max input tokens per API call (default: 250000 for GPT-5)\n  \n  // Audio Options\n  voiceId: string,           // Optional: ElevenLabs voice ID (default: Brian)\n  voiceSettings: object,     // Optional: Voice customization settings\n  \n  // Browser Options\n  headless: boolean,         // Optional: Run browser in headless mode (default: true)\n  enableProxy: boolean,      // Optional: Enable proxy (default: false)\n  proxyUrl: string,          // Optional: Proxy URL\n  proxyUsername: string,     // Optional: Proxy username\n  proxyPassword: string,     // Optional: Proxy password\n  proxyPoolSize: number      // Optional: Number of proxies in pool (default: 36)\n})\n```\n\n#### Methods\n\nAll methods return JSON objects with `{ success, ...data, error?, message? }` format.\n\n##### Processing Methods\n\n- **`processFile(filePath, asin?)`** - Process a PDF or EPUB file\n  - Returns: `{ success, basename, markdown, files, archive, hasAudio, asin, costs, message, error? }`\n  - Example:\n    ```javascript\n    const result = await forge.processFile('./book.pdf');\n    if (result.success) {\n      console.log('Archive:', result.archive);\n      console.log('Costs:', result.costs);\n    }\n    ```\n\n- **`processWebPage(url, outputDir?)`** - Process a web page URL\n  - Returns: `{ success, basename, dirName, markdown, files, directory, archive, hasAudio, url, title, costs, message, error? }`\n  - Example:\n    ```javascript\n    const result = await forge.processWebPage('https://example.com/article');\n    if (result.success) {\n      console.log('Summary:', result.markdown.substring(0, 100));\n    }\n    ```\n\n##### Search Methods\n\n- **`searchBookByTitle(title)`** - Search Amazon using Rainforest API\n  - Returns: `{ success, results, count, query, message, error? }`\n  - Example:\n    ```javascript\n    const result = await forge.searchBookByTitle('Clean Code');\n    if (result.success) {\n      console.log(`Found ${result.count} books`);\n    }\n    ```\n\n- **`searchAnnasArchive(query, options?)`** - Search Anna's Archive directly\n  - Returns: `{ success, results, count, query, options, message, error? }`\n  - Example:\n    ```javascript\n    const result = await forge.searchAnnasArchive('JavaScript', {\n      maxResults: 10,\n      format: 'pdf',\n      sortBy: 'date'\n    });\n    if (result.success) {\n      console.log(`Found ${result.count} results`);\n    }\n    ```\n\n- **`search1lib(query, options?)`** - Search 1lib.sk\n  - Returns: `{ success, results, count, query, options, message, error? }`\n\n##### Download Methods\n\n- **`downloadFromAnnasArchive(asin, outputDir?, bookTitle?)`** - Download from Anna's Archive\n  - Returns: `{ success, filepath, directory, asin, format, message, error? }`\n  - Example:\n    ```javascript\n    const result = await forge.downloadFromAnnasArchive('B075HYVHWK', '.');\n    if (result.success) {\n      console.log('Downloaded to:', result.filepath);\n    }\n    ```\n\n- **`downloadFrom1lib(bookUrl, outputDir?, bookTitle?, downloadUrl?)`** - Download from 1lib.sk\n  - Returns: `{ success, filepath, directory, title, format, message, error? }`\n\n- **`search1libAndDownload(query, searchOptions?, outputDir?, selectCallback?)`** - Search and download in one session\n  - Returns: `{ success, results, download, message, error? }`\n\n##### Generation Methods\n\n- **`generateSummary(pdfPath)`** - Generate AI summary from PDF\n  - Returns: `{ success, markdown, length, method, chunks?, message, error? }`\n  - Methods: `gpt5_pdf_upload`, `text_extraction_single`, `text_extraction_chunked`\n  - Example:\n    ```javascript\n    const result = await forge.generateSummary('./book.pdf');\n    if (result.success) {\n      console.log(`Generated ${result.length} char summary using ${result.method}`);\n    }\n    ```\n\n- **`generateAudioScript(markdown)`** - Generate audio-friendly narration script\n  - Returns: `{ success, script, length, message }`\n\n- **`generateAudio(text, outputPath)`** - Generate audio using ElevenLabs TTS\n  - Returns: `{ success, path, size, duration, message, error? }`\n\n- **`generateOutputFiles(markdown, basename, outputDir)`** - Generate all output formats\n  - Returns: `{ success, files: {...}, message }`\n\n##### Utility Methods\n\n- **`convertEpubToPdf(epubPath)`** - Convert EPUB to PDF\n  - Returns: `{ success, pdfPath, originalPath, message, error? }`\n\n- **`createBundle(files, archiveName)`** - Create tar.gz archive\n  - Returns: `{ success, path, files, message, error? }`\n\n- **`getCostSummary()`** - Get cost tracking information\n  - Returns: `{ success, openai, elevenlabs, rainforest, total, breakdown }`\n\n## Configuration\n\n### CLI Configuration (Recommended)\n\nFor CLI usage, run the setup command to configure your API keys:\n\n```bash\nsummary setup\n```\n\nThis saves your configuration to `~/.config/summary-forge/settings.json` so you don't need to manage environment variables.\n\n### Environment Variables (Alternative)\n\nFor programmatic usage or if you prefer environment variables, create a `.env` file:\n\n```env\nOPENAI_API_KEY=sk-your-key-here\nRAINFOREST_API_KEY=your-key-here\nELEVENLABS_API_KEY=sk-your-key-here  # Optional: for audio generation\nTWOCAPTCHA_API_KEY=your-key-here      # Optional: for CAPTCHA solving\nBROWSERLESS_API_KEY=your-key-here     # Optional\n\n# Browser Configuration\nHEADLESS=true                          # Run browser in headless mode\nENABLE_PROXY=false                     # Enable proxy for browser requests\nPROXY_URL=http://proxy.example.com    # Proxy URL (if enabled)\nPROXY_USERNAME=username                # Proxy username (if enabled)\nPROXY_PASSWORD=password                # Proxy password (if enabled)\nPROXY_POOL_SIZE=36                     # Number of proxies in your pool (default: 36)\n```\n\nOr set them in your shell:\n\n```bash\nexport OPENAI_API_KEY=sk-your-key-here\nexport RAINFOREST_API_KEY=your-key-here\nexport ELEVENLABS_API_KEY=sk-your-key-here  # Optional\n```\n\n### Configuration Priority\n\nWhen using the module programmatically, configuration is loaded in this order (highest priority first):\n\n1. **Constructor options** - Passed directly to `new SummaryForge(options)`\n2. **Environment variables** - From `.env` file or shell\n3. **Config file** - From `~/.config/summary-forge/settings.json` (CLI only)\n\n### Proxy Configuration (Recommended for Anna's Archive)\n\nTo avoid IP bans when downloading from Anna's Archive, configure a proxy during setup:\n\n```bash\nsummary setup\n```\n\nWhen prompted:\n1. Enable proxy: `Yes`\n2. Enter proxy URL: `http://your-proxy.com:8080`\n3. Enter proxy username and password\n\n**Why use a proxy?**\n- ✅ Avoids IP bans from Anna's Archive\n- ✅ USA-based proxies prevent geo-location issues\n- ✅ Works with both browser navigation and file downloads\n- ✅ Automatically applied to all download operations\n\n**Recommended Proxy Service:**\n\nWe recommend [Webshare.io](https://www.webshare.io/?referral_code=wwry9z1eiyjg) for reliable, USA-based proxies:\n- 🌎 USA-based IPs (no geo-location issues)\n- ⚡ Fast and reliable\n- 💰 Affordable pricing with free tier\n- 🔒 HTTP/HTTPS/SOCKS5 support\n\n**Important: Use Static Proxies for Sticky Sessions**\n\nFor Anna's Archive downloads, you need a **static/direct proxy** (not rotating) to maintain the same IP:\n\n1. In your Webshare dashboard, go to **Proxy** → **List**\n2. Copy a **Static Proxy** endpoint (not the rotating endpoint)\n3. Use the format: `http://host:port` (e.g., `http://45.95.96.132:8080`)\n4. Username format: `dmdgluqz-US-{session_id}` (session ID added automatically)\n\nThe tool automatically generates a unique session ID (1 to `PROXY_POOL_SIZE`) for each download to get a fresh IP, while maintaining that IP throughout the 5-10 minute download process.\n\n**Proxy Pool Size Configuration:**\n\nSet `PROXY_POOL_SIZE` to match your Webshare plan (default: 36):\n- Free tier: 10 proxies → `PROXY_POOL_SIZE=10`\n- Starter plan: 25 proxies → `PROXY_POOL_SIZE=25`\n- Professional plan: 100 proxies → `PROXY_POOL_SIZE=100`\n- Enterprise plan: 250+ proxies → `PROXY_POOL_SIZE=250`\n\nThe tool will randomly select a session ID from 1 to your pool size, distributing load across all available proxies.\n\n**Smart ISBN Detection:**\n\nWhen searching Anna's Archive, the tool automatically detects whether an identifier is a real ISBN or an Amazon ASIN:\n- **Real ISBNs** (10 or 13 numeric digits): Searches by ISBN for precise results\n- **Amazon ASINs** (alphanumeric): Searches by book title instead for better results\n- This ensures you get relevant search results even when Amazon returns proprietary ASINs instead of standard ISBNs\n\n**Note:** Rotating proxies (`p.webshare.io`) don't support sticky sessions. Use individual static proxy IPs from your proxy list instead.\n\n**Testing your proxy:**\n```bash\nnode test-proxy.js \u003cASIN\u003e\n```\n\nThis will verify your proxy configuration by attempting to download a book.\n\n### Audio Generation\n\nAudio generation is **optional** and requires an [ElevenLabs](https://try.elevenlabs.io/oh7kgotrpjnv) API key. If the key is not provided, the tool will skip audio generation and only create text-based outputs.\n\n**Get ElevenLabs API Key:** [Sign up here](https://try.elevenlabs.io/oh7kgotrpjnv) for high-quality text-to-speech.\n\n**Features:**\n- Uses ElevenLabs Turbo v2.5 model (optimized for audiobooks)\n- Default voice: Brian (best for technical content, customizable)\n- Automatically truncates long texts to fit API limits\n- Generates high-quality MP3 audio files\n- Natural, conversational narration style\n\n## Output\n\nThe tool generates:\n\n- `\u003cbook_name\u003e_summary.md` - Markdown summary\n- `\u003cbook_name\u003e_summary.txt` - Plain text summary\n- `\u003cbook_name\u003e_summary.pdf` - PDF summary with table of contents\n- `\u003cbook_name\u003e_summary.epub` - EPUB summary with clickable TOC\n- `\u003cbook_name\u003e_summary.mp3` - Audio summary (if ElevenLabs key provided)\n- `\u003cbook_name\u003e.pdf` - Original or converted PDF\n- `\u003cbook_name\u003e.epub` - Original EPUB (if input was EPUB)\n- `\u003cbook_name\u003e_bundle.tgz` - Compressed archive containing all files\n\n## Example Workflow\n\n```bash\n# 1. Search for a book\nsummary search\n# Enter: \"A Philosophy of Software Design\"\n# Select from results, get ASIN\n\n# 2. Download and process automatically\nsummary isbn B075HYVHWK\n# Downloads, asks if you want to process\n# Creates summary bundle automatically!\n\n# Alternative: Process a local file\nsummary file ~/Downloads/book.epub\n```\n\n## How It Works\n\n1. **Input Processing**: Accepts PDF or EPUB files (EPUB is converted to PDF)\n2. **Smart Processing Strategy**:\n   - **Small PDFs (\u003c400k chars)**: Direct upload to OpenAI's vision API\n   - **Large PDFs (\u003e400k chars)**: Intelligent chunking with synthesis\n3. **AI Summarization**: GPT-5 analyzes content with full formatting, tables, and diagrams\n4. **Format Conversion**: Uses Pandoc to convert the Markdown summary to PDF and EPUB\n5. **Audio Generation**: Optional TTS conversion using ElevenLabs\n6. **Bundling**: Creates a compressed archive with all generated files\n\n### Intelligent Chunking for Large PDFs\n\nFor PDFs exceeding 400,000 characters (typically 500+ pages), the tool automatically uses an intelligent chunking strategy:\n\n**How it works:**\n1. **Analysis**: Calculates optimal chunk size based on PDF statistics and GPT-5's token limits\n2. **Smart Token Management**: Respects GPT-5's 272k input token limit with safety margins\n3. **Page-Based Chunking**: Splits PDF into logical chunks that fit within token limits\n4. **Parallel Processing**: Each chunk is summarized independently by GPT-5\n5. **Intelligent Synthesis**: All chunk summaries are combined into a cohesive final summary\n6. **Quality Preservation**: Maintains narrative flow and eliminates redundancy\n\n**Token Limit Handling:**\n- **GPT-5 Input Limit**: 272,000 tokens\n- **System Overhead**: 20,000 tokens reserved for prompts and instructions\n- **Available Tokens**: 250,000 tokens for content\n- **Safety Margin**: 70% utilization to account for token estimation variance\n- **Chunk Size**: ~565,000 characters per chunk (based on 3.5 chars/token estimate)\n\n**Benefits:**\n- ✅ **Complete Coverage**: Processes entire books without truncation\n- ✅ **High Quality**: Each section gets full AI attention\n- ✅ **Seamless Output**: Final summary reads as a unified document\n- ✅ **Cost Efficient**: Optimizes token usage across multiple API calls\n- ✅ **Automatic**: No configuration needed - works transparently\n- ✅ **Token-Aware**: Respects API limits to prevent errors\n\n**Example Output:**\n```\n📊 PDF Stats: 523 pages, 1,245,678 chars, ~311,420 tokens\n📚 PDF is large - using intelligent chunking strategy\n   This will process the ENTIRE 523-page PDF without truncation\n📐 Using chunk size: 120,000 chars\n📦 Created 11 chunks for processing\n   Chunk 1: Pages 1-48 (119,234 chars)\n   Chunk 2: Pages 49-95 (118,901 chars)\n   ...\n✅ All 11 chunks processed successfully\n🔄 Synthesizing chunk summaries into final comprehensive summary...\n✅ Final summary synthesized: 45,678 characters\n```\n\n### Why Direct PDF Upload?\n\nThe tool prioritizes OpenAI's vision API for direct PDF upload when possible:\n\n- ✅ **Better Quality**: Preserves document formatting, tables, and diagrams\n- ✅ **More Accurate**: AI can see the actual PDF layout and structure\n- ✅ **Better for Technical Books**: Code examples and diagrams are preserved\n- ✅ **Fallback Strategy**: Automatically switches to intelligent chunking for large files\n\n## Testing\n\nSummary Forge includes a comprehensive test suite using Vitest.\n\n### Run Tests\n\n```bash\n# Run all tests\npnpm test\n\n# Run tests in watch mode\npnpm test:watch\n\n# Run tests with coverage report\npnpm test:coverage\n```\n\n### Test Coverage\n\nThe test suite includes:\n- ✅ 30+ passing tests\n- Constructor validation\n- Helper method tests\n- PDF upload functionality tests\n- API integration tests\n- Error handling tests\n- Edge case coverage\n- File operation tests\n\nSee [`test/summary-forge.test.js`](test/summary-forge.test.js) for the complete test suite.\n\n## Flashcard Generation\n\nSummary Forge includes powerful flashcard generation capabilities for study and review.\n\n### Printable PDF Flashcards\n\nGenerate double-sided flashcard PDFs optimized for printing:\n\n```javascript\nimport { extractFlashcards, generateFlashcardsPDF } from '@profullstack/summary-forge-module/flashcards';\nimport fs from 'node:fs/promises';\n\n// Read your markdown summary\nconst markdown = await fs.readFile('./book_summary.md', 'utf-8');\n\n// Extract Q\u0026A pairs\nconst extractResult = extractFlashcards(markdown, { maxCards: 50 });\nconsole.log(`Extracted ${extractResult.count} flashcards`);\n\n// Generate printable PDF\nconst pdfResult = await generateFlashcardsPDF(\n  extractResult.flashcards,\n  './flashcards.pdf',\n  {\n    title: 'JavaScript Fundamentals',\n    branding: 'SummaryForge.com',\n    cardWidth: 3.5,   // inches\n    cardHeight: 2.5,  // inches\n    fontSize: 11\n  }\n);\n\nconsole.log(`PDF created: ${pdfResult.path}`);\nconsole.log(`Total pages: ${pdfResult.pages}`);\n```\n\n### Individual Flashcard Images\n\nGenerate individual PNG images for each flashcard, perfect for web applications:\n\n```javascript\nimport { extractFlashcards, generateFlashcardImages } from '@profullstack/summary-forge-module/flashcards';\nimport fs from 'node:fs/promises';\n\n// Read your markdown summary\nconst markdown = await fs.readFile('./book_summary.md', 'utf-8');\n\n// Extract Q\u0026A pairs\nconst extractResult = extractFlashcards(markdown);\n\n// Generate individual PNG images\nconst imageResult = await generateFlashcardImages(\n  extractResult.flashcards,\n  './flashcards',  // Output directory\n  {\n    title: 'JavaScript Fundamentals',\n    branding: 'SummaryForge.com',\n    width: 800,   // pixels\n    height: 600,  // pixels\n    fontSize: 24\n  }\n);\n\nif (imageResult.success) {\n  console.log(`Generated ${imageResult.images.length} images`);\n  console.log('Files:', imageResult.images);\n  // Output: ['./flashcards/q-001.png', './flashcards/a-001.png', ...]\n}\n```\n\n**Image Naming Convention:**\n- `q-001.png`, `q-002.png`, etc. - Question cards\n- `a-001.png`, `a-002.png`, etc. - Answer cards\n\n**Use Cases:**\n- 🌐 Web-based flashcard applications\n- 📱 Mobile learning apps\n- 🎮 Interactive quiz games\n- 📊 Study progress tracking systems\n- 🔄 Spaced repetition software\n\n**Features:**\n- ✅ Clean, professional design with book title\n- ✅ Automatic text wrapping for long content\n- ✅ Customizable dimensions and styling\n- ✅ SVG-based rendering for crisp quality\n- ✅ Works in Docker (no native dependencies)\n\n### Flashcard Extraction Formats\n\nThe `extractFlashcards` function supports multiple markdown formats:\n\n**1. Explicit Q\u0026A Format:**\n```markdown\n**Q: What is a closure?**\nA: A closure is a function that has access to variables in its outer scope.\n```\n\n**2. Definition Lists:**\n```markdown\n**Closure**\n: A function that has access to variables in its outer scope.\n```\n\n**3. Question Headers:**\n```markdown\n### What is a closure?\n\nA closure is a function that has access to variables in its outer scope.\n```\n\n## Examples\n\nSee the [`examples/`](examples/) directory for more usage examples:\n\n- [`programmatic-usage.js`](examples/programmatic-usage.js) - Using as a module\n- [`flashcard-images-demo.js`](examples/flashcard-images-demo.js) - Generating flashcard images\n\n## Troubleshooting\n\n### Rate Limiting (1lib.sk)\n\nIf you encounter \"Too many requests\" errors from 1lib.sk:\n\n**Error Message:**\n```\nToo many requests from your IP xxx.xxx.xxx.xxx\nPlease wait 10 seconds. support@z-lib.fm. Err #ipd1\n```\n\n**Automatic Handling:**\nThe tool automatically detects rate limiting and:\n- ✅ Waits the requested time (usually 10 seconds)\n- ✅ Retries up to 3 times with exponential backoff\n- ✅ Adds a 2-second buffer to ensure rate limit has cleared\n\n**Manual Solutions:**\n1. **Wait a few minutes** before trying again\n2. **Use a different proxy session** (the tool rotates through your proxy pool automatically)\n3. **Switch to Anna's Archive**: `summary search \"book title\" --source anna`\n4. **Reduce concurrent requests** if running multiple downloads\n\n**Note:** The proxy pool helps distribute requests across different IPs, reducing rate limiting issues.\n\n### Download Button Not Found (1lib.sk)\n\nIf you encounter \"Download button not found\" errors when downloading from 1lib.sk:\n\n1. **Check Debug Files**: The tool automatically saves `debug-book-page.html` in the book's directory\n   - Open this file to inspect the actual page structure\n   - Look for download links or buttons that might have different selectors\n\n2. **Review Error Output**: The error message includes:\n   - All selectors that were tried\n   - List of links found on the page\n   - Location of the debug HTML file\n\n3. **Common Causes**:\n   - **Z-Access/Library Access Page**: Book page redirects to authentication page (most common)\n   - Page structure changed (1lib.sk updates their site)\n   - Book is deleted or unavailable\n   - Session expired or cookies not maintained\n   - Proxy issues preventing proper page load\n\n4. **Solutions**:\n   - **Recommended**: Use Anna's Archive instead: `summary search \"book title\" --source anna`\n   - Try the `search1lib` command separately to verify the book exists\n   - Check if the book page loads correctly in a regular browser with the same proxy\n   - Verify proxy configuration is working correctly\n   - Try a different book from search results\n\n5. **Known Issue - Z-Access Page**:\n   If you see links to `library-access.sk` or `Z-Access page` in the debug output, this means:\n   - The book page requires authentication or special access\n   - 1lib.sk's session management is blocking automated access\n   - **Workaround**: Use Anna's Archive which has better automation support\n\n**Example Debug Output (Z-Access Issue):**\n```\n❌ Download button not found on book page\n   Debug HTML saved to: ./uploads/book_name/debug-book-page.html\n   Found 6 links on page\n   First 5 links:\n   - https://library-access.sk (Z-Access page)\n   - mailto:blackbox@z-library.so (blackbox@z-library.so)\n   - https://www.reddit.com/r/zlibrary (https://www.reddit.com/r/zlibrary)\n```\n\n**Recommended Alternative:**\n```bash\n# Use Anna's Archive instead (more reliable for automation)\nsummary search \"prompt engineering\" --source anna\n```\n\n### IP Bans from Anna's Archive\n\nIf you're getting blocked by Anna's Archive:\n\n1. **Enable proxy** in your configuration:\n   ```bash\n   summary setup\n   ```\n   \n2. **Use a USA-based proxy** to avoid geo-location issues\n\n3. **Test your proxy** before downloading:\n   ```bash\n   node test-proxy.js B0BCTMXNVN\n   ```\n\n4. **Run browser in visible mode** to debug:\n   ```bash\n   summary config --headless false\n   ```\n\n### Proxy Configuration\n\nThe proxy is used for:\n- ✅ Browser navigation (Puppeteer)\n- ✅ File downloads (fetch with https-proxy-agent)\n- ✅ All HTTP requests to Anna's Archive\n\nSupported proxy formats:\n- `http://proxy.example.com:8080`\n- `https://proxy.example.com:8080`\n- `socks5://proxy.example.com:1080`\n- `http://proxy.example.com:8080-session-\u003cSESSION_ID\u003e` (sticky session)\n\n**Recommended Service:** [Webshare.io](https://www.webshare.io/?referral_code=wwry9z1eiyjg) - Reliable USA-based proxies with free tier available.\n\n**Webshare Sticky Sessions:**\nAdd `-session-\u003cYOUR_SESSION_ID\u003e` to your proxy URL to maintain the same IP:\n```\nhttp://p.webshare.io:80-session-myapp123\n```\n\n## CAPTCHA Solving\n\nWhen downloading from Anna's Archive, you may encounter CAPTCHAs. To automatically solve them:\n\n1. **Sign up for 2Captcha**: [Get API key here](https://2captcha.com/?from=9630996)\n2. **Add to configuration**:\n   ```bash\n   summary setup\n   ```\n3. **Enter your 2Captcha API key** when prompted\n\nThe tool will automatically detect and solve CAPTCHAs during downloads, making the process fully automated.\n\n## Limitations\n\n- Maximum PDF file size: No practical limit (intelligent chunking handles any size)\n- GPT-5 uses default temperature of 1 (not configurable)\n- Requires external tools: Calibre, Pandoc, XeLaTeX\n- CAPTCHA solving requires [2captcha.com](https://2captcha.com/?from=9630996) API key (optional)\n- Very large PDFs (1000+ pages) may incur higher API costs due to multiple chunk processing\n- Anna's Archive may block IPs without proxy configuration\n- Chunked processing uses text extraction (images/diagrams described in text only)\n\n## Roadmap\n\n- [x] ISBN/ASIN lookup via Anna's Archive\n- [x] Automatic download from Anna's Archive with CAPTCHA solving\n- [x] Book title search via Rainforest API\n- [x] CLI with interactive mode\n- [x] ESM module for programmatic use\n- [x] Audio generation with ElevenLabs TTS\n- [x] Direct PDF upload to OpenAI vision API\n- [x] EPUB format prioritization (open standard)\n- [ ] Support for more input formats (MOBI, AZW3)\n- [ ] Chunked processing for very large books (\u003e100MB)\n- [ ] Custom summary templates\n- [ ] Web interface\n- [ ] Multiple voice options for audio\n- [ ] Audio chapter markers\n- [ ] Batch processing multiple books\n\n## License\n\nISC\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprofullstack%2Fsummary-forge-module","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprofullstack%2Fsummary-forge-module","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprofullstack%2Fsummary-forge-module/lists"}