{"id":28257435,"url":"https://github.com/greigh/blockingmachine-core","last_synced_at":"2026-03-03T09:31:18.453Z","repository":{"id":289950854,"uuid":"972922554","full_name":"Greigh/blockingmachine-core","owner":"Greigh","description":"Core functionality for Blockingmachine","archived":false,"fork":false,"pushed_at":"2025-12-06T00:25:30.000Z","size":342,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-12-09T13:34:03.250Z","etag":null,"topics":["adblock","adguard","backend","open-source","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Greigh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-04-25T22:43:43.000Z","updated_at":"2025-12-06T00:25:34.000Z","dependencies_parsed_at":"2025-09-02T23:50:30.503Z","dependency_job_id":null,"html_url":"https://github.com/Greigh/blockingmachine-core","commit_stats":null,"previous_names":["greigh/blockingmachine-core"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/Greigh/blockingmachine-core","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Greigh%2Fblockingmachine-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Greigh%2Fblockingmachine-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Greigh%2Fblockingmachine-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Greigh%2Fblockingmachine-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Greigh","download_url":"https://codeload.github.com/Greigh/blockingmachine-core/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Greigh%2Fblockingmachine-core/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30039884,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-03T06:58:30.252Z","status":"ssl_error","status_checked_at":"2026-03-03T06:58:15.329Z","response_time":61,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adblock","adguard","backend","open-source","typescript"],"created_at":"2025-05-20T00:13:22.751Z","updated_at":"2026-03-03T09:31:18.073Z","avatar_url":"https://github.com/Greigh.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# @blockingmachine/core\n\nCore functionality for Blockingmachine, providing robust filter list processing and rule management for AdGuard Home and similar applications.\n\n[![LICENSE: BSD-3-Clause](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)\n[![GitHub Actions](https://github.com/greigh/blockingmachine-cli/workflows/CI/badge.svg)](https://github.com/greigh/blockingmachine-cli/actions)\n[![GitHub Release](https://img.shields.io/github/v/release/greigh/Blockingmachine-CLI)](https://github.com/greigh/Blockingmachine-core/releases)\n[![NPM VERSION](https://badge.fury.io/js/@blockingmachine%2Fcore.svg)](https://www.npmjs.com/package/@blockingmachine/core)\n[![NPM DOWNLOADS](https://img.shields.io/npm/dt/@blockingmachine/core)](https://www.npmjs.com/package/@blockingmachine/core)\n[![CODE SIZE](https://img.shields.io/github/languages/code-size/greigh/blockingmachine-core)](https://github.com/greigh/blockingmachine-core)\n[![COMMITS](https://img.shields.io/github/commit-activity/m/greigh/blockingmachine-core)](https://github.com/greigh/blockingmachine-core/graphs/commit-activity)\n\n## Related Projects\n\n- [Blockingmachine Desktop](https://github.com/greigh/Blockingmachine) - Desktop application\n- [Blockingmachine CLI](https://github.com/greigh/Blockingmachine-cli) - Command line interface\n- [Blockingmachine Database](https://github.com/greigh/Blockingmachine-database) - Filter list repository\n\n## Features\n\n- 🚀 Fast filter list processing\n- 🔄 Rule deduplication\n- 📥 Remote list fetching with retry logic\n- ✨ Clean rule formatting\n- 🛡️ AdGuard Home compatibility\n- 💪 TypeScript support\n- 🔁 Automatic retries for failed downloads\n- 🎯 Efficient memory usage\n- ⚡ Async processing support\n\n## Installation\n\n```bash\n# Using npm\nnpm install @blockingmachine/core\n\n# Using yarn\nyarn add @blockingmachine/core\n\n# Using pnpm\npnpm add @blockingmachine/core\n```\n\n## Quick Start\n\n```typescript\nimport {\n  RuleDeduplicator,\n  parseFilterList,\n  fetchContent,\n} from \"@blockingmachine/core\";\n\n// Basic usage\nconst rules = await parseFilterList(\"||example.com^\");\nconst deduplicator = new RuleDeduplicator();\nconst uniqueRules = deduplicator.process(rules);\n\n// Advanced usage with remote lists\nasync function processRemoteLists(urls: string[]) {\n  const deduplicator = new RuleDeduplicator();\n  let allRules: string[] = [];\n\n  for (const url of urls) {\n    const content = await fetchContent(url);\n    if (content) {\n      const rules = await parseFilterList(content);\n      allRules = [...allRules, ...rules];\n    }\n  }\n\n  return deduplicator.process(allRules);\n}\n```\n\n## API Reference\n\n### RuleDeduplicator\n\nProcess and deduplicate filtering rules.\n\n```typescript\nclass RuleDeduplicator {\n  constructor(options?: { caseSensitive?: boolean; keepComments?: boolean });\n\n  process(rules: string[]): string[];\n  addRule(rule: string): void;\n  clear(): void;\n}\n```\n\n#### Options\n\n- `caseSensitive` (boolean, default: true): Preserve case when comparing rules\n- `keepComments` (boolean, default: false): Retain comment lines in output\n\n### parseFilterList(content: string, options?: ParseOptions): Promise\u003cstring[]\u003e\n\nParse raw filter list content into individual rules.\n\n```typescript\ninterface ParseOptions {\n  skipComments?: boolean;\n  skipEmpty?: boolean;\n  trim?: boolean;\n}\n\n// Example usage\nconst rules = await parseFilterList(\"||example.com^\\n||example.org^\", {\n  skipComments: true,\n  skipEmpty: true,\n  trim: true,\n});\n```\n\n### fetchContent(url: string, options?: FetchOptions): Promise\u003cstring | null\u003e\n\nFetch remote filter lists with built-in retry logic.\n\n```typescript\ninterface FetchOptions {\n  timeout?: number;\n  retries?: number;\n  retryDelay?: number;\n}\n\n// Example with options\nconst content = await fetchContent(\"https://example.com/filterlist.txt\", {\n  timeout: 5000, // 5 seconds\n  retries: 3, // Try 3 times\n  retryDelay: 1000, // Wait 1 second between retries\n});\n```\n\n## Error Handling\n\n```typescript\ntry {\n  const content = await fetchContent(\"https://example.com/filterlist.txt\");\n  if (!content) {\n    console.error(\"Failed to fetch content\");\n    return;\n  }\n  const rules = await parseFilterList(content);\n} catch (error) {\n  console.error(\"Error processing rules:\", error);\n}\n```\n\n## Best Practices\n\n1. Memory Management\n\n```typescript\n// Process large lists in chunks\nconst deduplicator = new RuleDeduplicator();\nfor (const chunk of chunks) {\n  const rules = await parseFilterList(chunk);\n  deduplicator.process(rules);\n}\n```\n\n2. Error Recovery\n\n```typescript\n// Implement retry logic for failed fetches\nconst content = await fetchContent(url, {\n  retries: 5,\n  retryDelay: 2000,\n});\n```\n\n## Contributing\n\nWe welcome contributions! Please see our [contributing guidelines](CONTRIBUTING.md) for details.\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add some amazing feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n\n## License\n\nThis project is licensed under the BSD 3-Clause License - see the [LICENSE](LICENSE) file for details.\n\n### Summary of BSD 3-Clause License\n\nYou are free to:\n\n- Use the software commercially\n- Modify the software\n- Distribute the software\n- Place warranty on the software\n\nUnder the following conditions:\n\n- License and copyright notice must be included with the software\n- Neither the names of the copyright holder nor contributors may be used to promote derived products\n- Source code must retain copyright notice, list of conditions, and disclaimer\n\n[Read full license text](LICENSE)\n\n## Related Projects\n\n- [Blockingmachine Desktop](https://github.com/yourusername/Blockingmachine) - Desktop application\n- [Blockingmachine CLI](https://github.com/yourusername/blockingmachine-cli) - Command line interface\n\n## Support\n\n- 📝 [Documentation](https://github.com/yourusername/Blockingmachine/wiki)\n- 🐛 [Issue Tracker](https://github.com/yourusername/Blockingmachine/issues)\n- 💬 [Discussions](https://github.com/yourusername/Blockingmachine/discussions)\n\n## Acknowledgments\n\n- AdGuard for their excellent filter syntax documentation\n- All our contributors and users\n\n## Advanced Usage\n\n### Processing Multiple Filter Lists\n\n```typescript\nconst sources = [\n  \"https://example.com/list1.txt\",\n  \"https://example.com/list2.txt\",\n];\n\nconst processAllLists = async () =\u003e {\n  const deduplicator = new RuleDeduplicator();\n  const results = await Promise.allSettled(\n    sources.map((url) =\u003e fetchContent(url)),\n  );\n\n  for (const result of results) {\n    if (result.status === \"fulfilled\" \u0026\u0026 result.value) {\n      const rules = await parseFilterList(result.value);\n      deduplicator.process(rules);\n    }\n  }\n};\n```\n\n### Custom Rule Processing\n\n```typescript\nconst customProcessor = async (content: string) =\u003e {\n  const rules = await parseFilterList(content, {\n    skipComments: true,\n    skipEmpty: true,\n    trim: true,\n  });\n\n  // Custom processing logic\n  return rules.filter((rule) =\u003e {\n    // Filter out rules containing specific patterns\n    return !rule.includes(\"specific-pattern\");\n  });\n};\n```\n\n## Performance Tips\n\n### Memory Optimization\n\n- Process large lists in chunks\n- Clear the deduplicator cache periodically\n- Use streaming for very large files\n\n```typescript\nconst processLargeFile = async (filePath: string) =\u003e {\n  const deduplicator = new RuleDeduplicator();\n  const CHUNK_SIZE = 1000;\n  let rules: string[] = [];\n\n  // Read file in chunks\n  for await (const chunk of readFileInChunks(filePath)) {\n    const parsedRules = await parseFilterList(chunk);\n    rules = [...rules, ...deduplicator.process(parsedRules)];\n\n    if (rules.length \u003e CHUNK_SIZE) {\n      // Process chunk and clear cache\n      await processRules(rules);\n      deduplicator.clear();\n      rules = [];\n    }\n  }\n};\n```\n\n### Concurrent Processing\n\n```typescript\nconst concurrentProcessing = async (urls: string[]) =\u003e {\n  const BATCH_SIZE = 5; // Process 5 URLs at a time\n  const results: string[] = [];\n\n  for (let i = 0; i \u003c urls.length; i += BATCH_SIZE) {\n    const batch = urls.slice(i, i + BATCH_SIZE);\n    const batchResults = await Promise.all(\n      batch.map((url) =\u003e fetchContent(url)),\n    );\n    results.push(...batchResults.filter(Boolean));\n  }\n\n  return results;\n};\n```\n\n## Debugging\n\nEnable debug logging by setting the environment variable:\n\n```bash\n# macOS/Linux\nexport DEBUG=blockingmachine:*\n\n# In your code\nconst debug = require('debug')('blockingmachine:core');\ndebug('Processing rules:', rules.length);\n```\n\n## Common Issues and Solutions\n\n### Timeout Issues\n\n```typescript\n// Increase timeout for slow connections\nconst content = await fetchContent(url, {\n  timeout: 10000, // 10 seconds\n  retries: 5,\n});\n```\n\n### Memory Issues\n\n```typescript\n// Use streaming API for large files\nconst deduplicator = new RuleDeduplicator({\n  useStreaming: true,\n  chunkSize: 1000,\n});\n```\n\n### Rate Limiting\n\n```typescript\n// Add delays between requests\nconst delay = (ms: number) =\u003e new Promise((resolve) =\u003e setTimeout(resolve, ms));\n\nconst fetchWithRateLimit = async (urls: string[]) =\u003e {\n  for (const url of urls) {\n    await fetchContent(url);\n    await delay(1000); // Wait 1 second between requests\n  }\n};\n```\n\n## FAQ\n\n### General Questions\n\n**Q: What types of filter lists are supported?**\nA: We support AdGuard-style filter lists, including:\n\n- Domain-based rules (`||example.com^`)\n- Basic pattern rules (`/ads/`)\n- Comment lines (`! This is a comment`)\n- AdGuard Home specific syntax\n\n**Q: How large of a filter list can this handle?**\nA: The library is optimized for large lists and can handle millions of rules when used with proper memory management practices (see Performance Tips section).\n\n### Performance\n\n**Q: Why is processing taking longer than expected?**\nA: Several factors can affect processing speed:\n\n- Large number of rules\n- Complex pattern matching\n- Network latency when fetching remote lists\n- System memory constraints\n\nSolution: Use the chunking and streaming options described in the Performance Tips section.\n\n### Common Use Cases\n\n**Q: How do I combine multiple filter lists?**\n\n```typescript\nconst combineLists = async (urls: string[]) =\u003e {\n  const deduplicator = new RuleDeduplicator();\n  for (const url of urls) {\n    const content = await fetchContent(url);\n    if (content) {\n      const rules = await parseFilterList(content);\n      deduplicator.process(rules);\n    }\n  }\n  return deduplicator.process([]);\n};\n```\n\n**Q: How can I exclude certain domains from being blocked?**\n\n```typescript\nconst excludeDomains = (rules: string[], excludeList: string[]) =\u003e {\n  return rules.filter((rule) =\u003e {\n    return !excludeList.some((domain) =\u003e rule.includes(domain));\n  });\n};\n```\n\n### Troubleshooting\n\n**Q: Why am I getting timeout errors?**\nA: Remote lists might be slow to respond. Try:\n\n```typescript\nconst content = await fetchContent(url, {\n  timeout: 30000, // 30 seconds\n  retries: 5, // 5 attempts\n  retryDelay: 2000, // 2 seconds between retries\n});\n```\n\n**Q: How do I handle invalid rules?**\nA: Use the parsing options to skip problematic rules:\n\n```typescript\nconst rules = await parseFilterList(content, {\n  skipInvalid: true,\n  onError: (error, rule) =\u003e {\n    console.warn(`Skipping invalid rule: ${rule}`);\n  },\n});\n```\n\n### Integration\n\n**Q: Can I use this with Express.js?**\nA: Yes, here's a basic example:\n\n```typescript\nimport express from \"express\";\nimport { RuleDeduplicator, parseFilterList } from \"@/core\";\n\nconst app = express();\n\napp.post(\"/process-rules\", async (req, res) =\u003e {\n  try {\n    const rules = await parseFilterList(req.body.content);\n    const deduplicator = new RuleDeduplicator();\n    const processed = deduplicator.process(rules);\n    res.json({ rules: processed });\n  } catch (error) {\n    res.status(500).json({ error: error.message });\n  }\n});\n```\n\n**Q: How do I save processed rules to a file?**\nA: Use the built-in file system functions:\n\n```typescript\nimport { promises as fs } from \"fs\";\n\nconst saveRules = async (rules: string[], filepath: string) =\u003e {\n  await fs.writeFile(filepath, rules.join(\"\\n\"), \"utf8\");\n};\n```\n\n### Updates and Maintenance\n\n**Q: How often should I update my filter lists?**\nA: Best practices suggest:\n\n- Daily updates for actively maintained lists\n- Weekly updates for stable lists\n- Implement rate limiting when fetching multiple lists\n- Use the `If-Modified-Since` header (supported by `fetchContent`)\n\n**Q: How do I handle updates efficiently?**\nA: Use the incremental update feature:\n\n```typescript\nconst updateRules = async (existingRules: string[], newContent: string) =\u003e {\n  const newRules = await parseFilterList(newContent);\n  const deduplicator = new RuleDeduplicator();\n  return deduplicator.process([...existingRules, ...newRules]);\n};\n```\n\n## Timeline\n\n### Current Release (v1.0.0-beta.3)\n\n- 🎉 Initial public release\n- 🚀 Core functionality implementation\n- 💪 TypeScript support\n- 📥 Remote list fetching\n- 🔄 Rule deduplication\n- ⚡ Async processing\n- 🖥️ Works with Blockingmachine Desktop\n- ✅ Improved compatibility with Blockingmachine CLI\n- 🐞 Bug fixes and performance improvements\n\n### Upcoming Features (v1.0.0)\n\n- 📊 Rule statistics and analytics\n- 🔍 Enhanced pattern matching\n- 📋 Support for additional filter list formats\n- 🌐 Better network resilience\n- 🎯 Rule optimization algorithms\n- 📦 Reduced bundle size\n- 🧪 Extended test coverage\n\n### Future Roadmap (v1.x+)\n\n- 🔄 Streaming API for large files\n- 🌍 Internationalization support\n- 🔒 Enhanced security features\n- 📈 Performance improvements\n- 🧩 Plugin system\n- 🤝 Third-party integrations\n- 🛠️ Improved error handling\n\n### Version History\n\n#### 1.0.0-beta.3 (Current)\n\n- Improved CLI compatibility\n- Fixed several critical bugs\n- Performance optimizations\n- Documentation updates\n- Added more comprehensive examples\n- Fixed issues with NPM package integration\n\n#### 1.0.0-beta.2\n\n- Added support for additional filter formats\n- Enhanced error handling\n- Improved documentation\n- Fixed dependency issues\n- Fixed CLI integration bugs\n\n#### 1.0.0-beta.1\n\n- Initial public release\n- Core functionality stable\n- Basic documentation\n- Essential features implemented\n\n#### 0.9.0 (Internal)\n\n- Feature complete\n- Internal testing\n- Performance optimization\n- Documentation drafting\n\n#### 0.5.0 (Development)\n\n- Core architecture\n- Basic feature implementation\n- Initial testing setup\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgreigh%2Fblockingmachine-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgreigh%2Fblockingmachine-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgreigh%2Fblockingmachine-core/lists"}