{"id":34042533,"url":"https://github.com/riodevnet/snakyscraper","last_synced_at":"2026-04-01T23:01:35.453Z","repository":{"id":310069249,"uuid":"1005931522","full_name":"riodevnet/snakyscraper","owner":"riodevnet","description":"SnakyScraper is a lightweight and Pythonic web scraping toolkit built on top of BeautifulSoup and Requests. It provides an elegant interface for extracting structured HTML and metadata from websites with clean, direct outputs.","archived":false,"fork":false,"pushed_at":"2025-08-15T08:00:13.000Z","size":4,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-28T00:42:33.153Z","etag":null,"topics":["python","scraper","scraping"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/snakyscraper/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/riodevnet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-21T05:39:14.000Z","updated_at":"2025-08-18T05:08:51.000Z","dependencies_parsed_at":"2025-08-15T15:40:45.582Z","dependency_job_id":"0b8f8fbf-79b7-49ed-bccd-3a0dd1f059eb","html_url":"https://github.com/riodevnet/snakyscraper","commit_stats":null,"previous_names":["riodevnet/snakyscraper"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/riodevnet/snakyscraper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riodevnet%2Fsnakyscraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riodevnet%2Fsnakyscraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riodevnet%2Fsnakyscraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riodevnet%2Fsnakyscraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/riodevnet","download_url":"https://codeload.github.com/riodevnet/snakyscraper/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/riodevnet%2Fsnakyscraper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31292781,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T21:15:39.731Z","status":"ssl_error","status_checked_at":"2026-04-01T21:15:34.046Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","scraper","scraping"],"created_at":"2025-12-13T22:33:56.430Z","updated_at":"2026-04-01T23:01:35.444Z","avatar_url":"https://github.com/riodevnet.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# 🐍 SnakyScraper\n\n**SnakyScraper** is a lightweight and Pythonic web scraping toolkit built on top of BeautifulSoup and Requests. It provides an elegant interface for extracting structured HTML and metadata from websites with clean, direct outputs.\n\n\u003e Fast. Accurate. Snake-style scraping. 🐍🎯\n\n---\n\n## 🚀 Features\n\n- ✅ Extract metadata: title, description, keywords, author, and more\n- ✅ Built-in support for Open Graph, Twitter Card, canonical, and CSRF tags\n- ✅ Extract HTML structures: `h1`–`h6`, `p`, `ul`, `ol`, `img`, links\n- ✅ Powerful `filter()` method with class, ID, and tag-based selectors\n- ✅ `return_html` toggle to return clean text or raw HTML\n- ✅ Simple return values: string, list, or dictionary\n- ✅ Powered by BeautifulSoup4 and Requests\n\n---\n\n## 📦 Installation\n\n```bash\npip install snakyscraper\n```\n\n\u003e Requires Python 3.7 or later\n\n---\n\n## 🛠️ Basic Usage\n\n```python\nfrom snakyscraper import SnakyScraper\n\nscraper = SnakyScraper(\"https://example.com\")\n\n# Get the page title\nprint(scraper.title())  # \"Welcome to Example.com\"\n\n# Get meta description\nprint(scraper.description())  # \"This is the example meta description.\"\n\n# Get all \u003ch1\u003e elements\nprint(scraper.h1())  # [\"Welcome\", \"Latest News\"]\n\n# Extract Open Graph metadata\nprint(scraper.open_graph())  # {\"og:title\": \"...\", \"og:description\": \"...\", ...}\n\n# Custom filter: find all div.card elements and extract child tags\nprint(scraper.filter(\n    element=\"div\",\n    attributes={\"class\": \"card\"},\n    multiple=True,\n    extract=[\"h1\", \"p\", \".title\", \"#desc\"]\n))\n```\n\n---\n\n## 🧪 Available Methods\n\n### 🔹 Page Metadata\n\n```python\nscraper.title()\nscraper.description()\nscraper.keywords()\nscraper.keyword_string()\nscraper.charset()\nscraper.canonical()\nscraper.content_type()\nscraper.author()\nscraper.csrf_token()\nscraper.image()\n```\n\n### 🔹 Open Graph \u0026 Twitter Card\n\n```python\nscraper.open_graph()\nscraper.open_graph(\"og:title\")\n\nscraper.twitter_card()\nscraper.twitter_card(\"twitter:title\")\n```\n\n### 🔹 Headings \u0026 Text\n\n```python\nscraper.h1()\nscraper.h2()\nscraper.h3()\nscraper.h4()\nscraper.h5()\nscraper.h6()\nscraper.p()\n```\n\n### 🔹 Lists\n\n```python\nscraper.ul()\nscraper.ol()\n```\n\n### 🔹 Images\n\n```python\nscraper.images()\nscraper.image_details()\n```\n\n### 🔹 Links\n\n```python\nscraper.links()\nscraper.link_details()\n```\n\n---\n\n## 🔍 Custom DOM Filtering\n\nUse `filter()` to target specific DOM elements and extract nested content.\n\n#### ▸ Single element\n\n```python\nscraper.filter(\n    element=\"div\",\n    attributes={\"id\": \"main\"},\n    multiple=False,\n    extract=[\".title\", \"#description\", \"p\"]\n)\n```\n\n#### ▸ Multiple elements\n\n```python\nscraper.filter(\n    element=\"div\",\n    attributes={\"class\": \"card\"},\n    multiple=True,\n    extract=[\"h1\", \".subtitle\", \"#meta\"]\n)\n```\n\n\u003e The `extract` argument accepts tag names, class selectors (e.g., `.title`), or ID selectors (e.g., `#meta`).  \n\u003e Output keys are automatically normalized:  \n\u003e `.title` → `class__title`, `#meta` → `id__meta`\n\n#### ▸ Clean Text Output\n\nYou can also disable raw HTML output:\n\n```python\nscraper.filter(\n    element=\"p\",\n    attributes={\"class\": \"dark-text\"},\n    multiple=True,\n    return_html=False\n)\n```\n\n---\n\n## 📦 Output Example\n\n```python\nscraper.title()\n# \"Welcome to Example.com\"\n\nscraper.h1()\n# [\"Main Heading\", \"Another Title\"]\n\nscraper.open_graph(\"og:title\")\n# \"Example OG Title\"\n```\n\n---\n\n## 🤝 Contributing\n\nContributions are welcome!  \nFound a bug or want to request a feature? Please open an [issue](https://github.com/riodevnet/snakyscraper/issues) or submit a pull request.\n\n---\n\n## 📄 License\n\nMIT License © 2025 — SnakyScraper\n\n---\n\n## 🔗 Related Projects\n\n- [BeautifulSoup4](https://www.crummy.com/software/BeautifulSoup/)\n- [Requests](https://docs.python-requests.org/)\n- [lxml](https://lxml.de/)\n\n---\n\n## 💡 Why SnakyScraper?\n\n\u003e Think of it as your Pythonic sniper — targeting HTML content with precision and elegance.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Friodevnet%2Fsnakyscraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Friodevnet%2Fsnakyscraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Friodevnet%2Fsnakyscraper/lists"}