{"id":13726528,"url":"https://github.com/beb7/gflare-tk","last_synced_at":"2025-05-07T21:33:02.446Z","repository":{"id":43227329,"uuid":"262570179","full_name":"beb7/gflare-tk","owner":"beb7","description":"Open-Source Python Based SEO Web Crawler","archived":false,"fork":false,"pushed_at":"2023-07-07T13:58:18.000Z","size":40866,"stargazers_count":159,"open_issues_count":17,"forks_count":19,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-10-03T06:54:30.151Z","etag":null,"topics":["crawler","python","robots-txt","scraper","seo","seo-crawler","tkinter"],"latest_commit_sha":null,"homepage":"https://greenflare.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/beb7.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-05-09T12:52:04.000Z","updated_at":"2024-10-01T20:23:35.000Z","dependencies_parsed_at":"2024-01-10T19:10:26.156Z","dependency_job_id":null,"html_url":"https://github.com/beb7/gflare-tk","commit_stats":{"total_commits":733,"total_committers":5,"mean_commits":146.6,"dds":0.03137789904502042,"last_synced_commit":"59275bd49c5b9d68759ce0a507e4d57abb66e1af"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/beb7%2Fgflare-tk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/beb7%2Fgflare-tk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/beb7%2Fgflare-tk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/beb7%2Fgflare-tk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/beb7","download_url":"https://codeload.github.com/beb7/gflare-tk/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224654306,"owners_count":17347721,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","python","robots-txt","scraper","seo","seo-crawler","tkinter"],"created_at":"2024-08-03T01:03:10.595Z","updated_at":"2024-11-14T16:34:02.600Z","avatar_url":"https://github.com/beb7.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Greenflare SEO Web Crawler\n[![PyPI version](https://badge.fury.io/py/greenflare.svg)](https://badge.fury.io/py/greenflare)\n[![Supported Python Versions](https://img.shields.io/pypi/pyversions/greenflare.svg)](https://img.shields.io/pypi/pyversions/greenflare.svg)\n[![Downloads](https://pepy.tech/badge/greenflare)](https://pepy.tech/project/greenflare)\n[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)\n\nGreenflare is a lightweight free and open-source SEO web crawler for Linux, Mac, and Windows, and is dedicated to delivering high quality \nSEO insights and analysis solutions to the world.\n\n## Features\n\n* Cross-platform (Linux, Mac, and Windows)\n* Low hardware requirements\n* Scalable (tested against sites with 4M+ URLs) \n* Reports on on-page SEO elements (i.e. page title, meta robots, canonical tag)\n* Analysis of HTTP header responses (i.e. X-Robots-Tag, Canonical HTTP Header)\n* Status code reporting (i.e. 301, 404, 503 etc.) \n* robots.txt parser (implemented against the suggested REP standard by Google)\n* Custom extraction through XPath or CSS\n* Custom exclusion of URLs through various patterns\n* Quick filtering and sorting of crawl data\n* View broken internal links (3xx, 4xx, 5xx)\n* Greenflare databases (.gflaredb) are sqlite tables \n* Export any view to CSV\n\n\n## Getting Started\n\nThe quickest way to get started using Greenflare is to download one of \nour pre-built installers. Choose the version for your OS from our Download page:\n\nhttps://greenflare.io/download\n\n## Python Package\n\nGreenflare is also available as a pypi package:\n\n`pip install greenflare`\n\nThe use of a virtual environment (venv) is recommended. \nLinux users may chose to install ttkthemes for an improved visual experience.  \n\n\n## Developers\n\nAre you interested in becoming more involved in the development of \nGreenflare? Please submit a pull request if you want to help to build new amazing features or to fix nasty bugs!\nAlternatively, please email ben at greenflare dot io\n\n## Report a bug\n\nPlease report bugs by creating a new issue directly on GitHub:\n\nhttps://github.com/beb7/gflare-tk/issues\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbeb7%2Fgflare-tk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbeb7%2Fgflare-tk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbeb7%2Fgflare-tk/lists"}