{"id":50087309,"url":"https://github.com/plasmate-labs/plasmate-python","last_synced_at":"2026-07-09T22:30:23.443Z","repository":{"id":363312337,"uuid":"1191597831","full_name":"plasmate-labs/plasmate-python","owner":"plasmate-labs","description":"Python SDK for Plasmate - fetch web pages as structured SOM JSON. pip install plasmate.","archived":false,"fork":false,"pushed_at":"2026-03-26T18:45:41.000Z","size":14,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-06-08T12:04:59.085Z","etag":null,"topics":["ai-agents","headless-browser","llm","plasmate","python","sdk","som","web-scraping"],"latest_commit_sha":null,"homepage":"https://docs.plasmate.app/sdk-python","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/plasmate-labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-25T12:01:44.000Z","updated_at":"2026-03-26T18:45:45.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/plasmate-labs/plasmate-python","commit_stats":null,"previous_names":["plasmate-labs/plasmate-python"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/plasmate-labs/plasmate-python","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plasmate-labs%2Fplasmate-python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plasmate-labs%2Fplasmate-python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plasmate-labs%2Fplasmate-python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plasmate-labs%2Fplasmate-python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/plasmate-labs","download_url":"https://codeload.github.com/plasmate-labs/plasmate-python/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plasmate-labs%2Fplasmate-python/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35314872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-09T02:00:07.329Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agents","headless-browser","llm","plasmate","python","sdk","som","web-scraping"],"created_at":"2026-05-22T21:00:29.249Z","updated_at":"2026-07-09T22:30:23.433Z","avatar_url":"https://github.com/plasmate-labs.png","language":"Python","funding_links":[],"categories":["Web Scraping"],"sub_categories":[],"readme":"# Plasmate Python Toolkit\n\n**Simple, powerful web scraping with [Plasmate](https://plasmate.dev)'s Semantic Object Model.**\n\nThis toolkit provides a clean Python interface to the Plasmate CLI for developers who want structured web data without the overhead of a full framework like Scrapy.\n\n## The Problem\n\nTraditional scraping with libraries like `requests` and `BeautifulSoup` requires you to write complex, site-specific parsers that break whenever a CSS class or HTML tag changes.\n\n```python\n# The old way: brittle, complex, high-maintenance\nimport requests\nfrom bs4 import BeautifulSoup\n\nurl = 'https://news.ycombinator.com/'\nresponse = requests.get(url)\nsoup = BeautifulSoup(response.text, 'html.parser')\n\nstories = []\nfor row in soup.select('tr.athing'):\n    title_el = row.select_one('td.title \u003e span.titleline \u003e a')\n    stories.append({\n        'title': title_el.text,\n        'url': title_el.get('href'),\n    })\n```\n\n**Result**: You get fragile code and raw HTML. Processing this with an LLM is expensive: a typical news homepage can be **10,000-20,000 tokens**.\n\n## The Solution\n\nPlasmate handles fetching and parsing, giving you a clean **Semantic Object Model (SOM)**. This library makes it trivial to use Plasmate in any Python script.\n\n```python\n# The new way: simple, robust, low-maintenance\nfrom plasmate_scraper import fetch, extract_links\n\nsom = fetch(\"https://news.ycombinator.com/\")\nlinks = extract_links(som)\n\nfor link in links:\n    print(f\"{link['text']} -\u003e {link['url']}\")\n```\n\n**Result**: You get clean data and a massive reduction in token usage for LLM pipelines. The same homepage as a Plasmate SOM is often **1,000-2,000 tokens** — an order of magnitude smaller.\n\n## Installation\n\n```bash\npip install plasmate-scraper\n```\n\nYou also need the Plasmate CLI:\n\n```bash\n# macOS\nbrew install nicholasgasior/plasmate/plasmate\n\n# From source\ncargo install plasmate\n\n# Or download from https://github.com/nicholasgasior/plasmate/releases\n```\n\n## Quick Start\n\n### Fetch a single page\n\n```python\nfrom plasmate_scraper import fetch, extract_text\n\n# Fetch the page and get the SOM\nsom = fetch(\"https://example.com\")\n\n# The SOM is a dictionary\nprint(som['title'])\n# \u003e Example Domain\n\n# Use helpers to extract common data\ntext = extract_text(som)\nprint(text)\n# \u003e Example Domain\n# \u003e This domain is for use in illustrative examples.\n# \u003e More information...\n```\n\n### Fetch multiple pages concurrently\n\n```python\nfrom plasmate_scraper import batch_fetch\n\nurls = [\n    \"https://news.ycombinator.com\",\n    \"https://github.com/explore\",\n    \"https://dev.to\",\n]\n\n# Fetches all pages in parallel\nresults = batch_fetch(urls, max_concurrent=10)\n\nfor som in results:\n    if 'error' in som:\n        print(f\"Failed to fetch {som['url']}: {som['error']}\")\n    else:\n        print(f\"Fetched: {som.get('title', 'Untitled')}\")\n```\n\n## API Reference\n\n### `fetch()`\n\n```python\nfetch(\n    url: str,\n    *,\n    timeout: int = 30,\n    javascript: bool = True,\n    format: str = \"json\",\n    binary: str = \"plasmate\",\n    extra_args: list[str] | None = None,\n) -\u003e dict\n```\n\nFetches a URL and returns the parsed SOM. Raises `PlasmateError` on failure.\n\n### `batch_fetch()`\n\n```python\nbatch_fetch(\n    urls: list[str],\n    *,\n    max_concurrent: int = 5,\n    raise_on_error: bool = False,\n    # ... accepts same args as fetch()\n) -\u003e list[dict]\n```\n\nFetches multiple URLs in parallel. If `raise_on_error` is `False` (default), failures are returned as dicts like `{'url': '...', 'error': '...'}`.\n\n### Utility Functions\n\nAll utilities take a SOM dictionary as input.\n\n```python\nfrom plasmate_scraper import (\n    extract_text,      # All text content as a string\n    extract_links,     # [{'url': '...', 'text': '...'}]\n    extract_headings,  # [{'level': 1, 'text': '...'}]\n    extract_tables,    # Table regions/elements from the SOM\n    extract_images,    # [{'src': '...', 'alt': '...'}]\n    extract_by_role,   # Filter elements by SOM role\n)\n```\n\n## Comparison to Alternatives\n\n| Feature | `requests` + `bs4` | `playwright` | `plasmate-scraper` |\n|---------|----------------------|--------------|--------------------|\n| Parsing | Manual (CSS/XPath) | Manual (CSS/XPath) | **Automatic (SOM)** |\n| Resilience | Low (breaks easily) | Low (breaks easily) | **High (semantic)** |\n| JS Support | No | Yes | **Yes (default)** |\n| Concurrency | Manual (e.g., `ThreadPool`) | Manual | **Built-in (`batch_fetch`)** |\n| LLM-ready | No (too verbose) | No (too verbose) | **Yes (token-efficient)** |\n\n## License\n\nApache 2.0 — see [LICENSE](LICENSE).\n\n\n---\n\n## Part of the Plasmate Ecosystem\n\n| | |\n|---|---|\n| **Engine** | [plasmate](https://github.com/plasmate-labs/plasmate) - The browser engine for agents |\n| **MCP** | [plasmate-mcp](https://github.com/plasmate-labs/plasmate-mcp) - Claude Code, Cursor, Windsurf |\n| **Extension** | [plasmate-extension](https://github.com/plasmate-labs/plasmate-extension) - Chrome cookie export |\n| **SDKs** | [Python](https://github.com/plasmate-labs/plasmate-python) / [Node.js](https://github.com/plasmate-labs/quickstart-node) / [Go](https://docs.plasmate.app/sdk-go) / [Rust](https://github.com/plasmate-labs/quickstart-rust) |\n| **Frameworks** | [LangChain](https://github.com/langchain-ai/langchain/pull/36208) / [CrewAI](https://github.com/plasmate-labs/crewai-plasmate) / [AutoGen](https://github.com/plasmate-labs/autogen-plasmate) / [Smolagents](https://github.com/plasmate-labs/smolagents-plasmate) |\n| **Tools** | [Scrapy](https://github.com/plasmate-labs/scrapy-plasmate) / [Audit](https://github.com/plasmate-labs/plasmate-audit) / [A11y](https://github.com/plasmate-labs/plasmate-a11y) / [GitHub Action](https://github.com/plasmate-labs/som-action) |\n| **Resources** | [Awesome Plasmate](https://github.com/plasmate-labs/awesome-plasmate) / [Notebooks](https://github.com/plasmate-labs/notebooks) / [Benchmarks](https://github.com/plasmate-labs/plasmate-benchmarks) |\n| **Docs** | [docs.plasmate.app](https://docs.plasmate.app) |\n| **W3C** | [Web Content Browser for AI Agents](https://www.w3.org/community/web-content-browser-ai/) |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplasmate-labs%2Fplasmate-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fplasmate-labs%2Fplasmate-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplasmate-labs%2Fplasmate-python/lists"}