{"id":30593364,"url":"https://github.com/techwithty/ahrefs","last_synced_at":"2025-08-29T18:13:39.194Z","repository":{"id":309618430,"uuid":"1036960299","full_name":"TechWithTy/ahrefs","owner":"TechWithTy","description":"Ahrefs internal Python SDK and FastAPI API integration with retries, rate limiting, typed handlers, and centralized routing for Site Explorer, Keywords Explorer, Rank Tracker, Management, and more.","archived":false,"fork":false,"pushed_at":"2025-08-12T21:05:04.000Z","size":18,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-08-12T23:21:49.139Z","etag":null,"topics":["ahrefs","api","fastapi","internal-tools","observability","production-ready","python","rate-limiting","retries","sdk","seo","typed"],"latest_commit_sha":null,"homepage":"https://www.cybershoptech.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TechWithTy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-12T21:02:18.000Z","updated_at":"2025-08-12T21:05:07.000Z","dependencies_parsed_at":"2025-08-12T23:21:54.473Z","dependency_job_id":"8deb07fa-c542-41dd-8046-cf25f4b9a6a6","html_url":"https://github.com/TechWithTy/ahrefs","commit_stats":null,"previous_names":["techwithty/ahrefs"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/TechWithTy/ahrefs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TechWithTy%2Fahrefs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TechWithTy%2Fahrefs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TechWithTy%2Fahrefs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TechWithTy%2Fahrefs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TechWithTy","download_url":"https://codeload.github.com/TechWithTy/ahrefs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TechWithTy%2Fahrefs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272735311,"owners_count":24984283,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-29T02:00:10.610Z","response_time":87,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ahrefs","api","fastapi","internal-tools","observability","production-ready","python","rate-limiting","retries","sdk","seo","typed"],"created_at":"2025-08-29T18:13:36.372Z","updated_at":"2025-08-29T18:13:39.184Z","avatar_url":"https://github.com/TechWithTy.png","language":"Python","readme":"# Ahrefs Internal SDK + FastAPI API\n\nTyped, production-ready internal Python SDK and FastAPI API for Ahrefs. Includes retries, rate limiting, clear error handling, and centralized routes with category-based handlers.\n\n- SDK path: `backend/app/core/landing_page/ahrefs/`\n- API path: `backend/app/core/landing_page/ahrefs/api/`\n- Public endpoints covered: Site Explorer groups, Keywords Explorer, Rank Tracker, Overview (incl. competitors pages), SERP Overview, Batch Analysis, Subscription, Management (projects/keywords/competitors/locations-languages/keyword lists), and Public Crawler IPs.\n\n## Features\n\n- Typed request models and handler functions per category\n- Centralized FastAPI routes with DI-based client access\n- Configurable auth via header or query param\n- Retries with backoff and robust error mapping\n- Lightweight token/leaky-bucket rate limiter\n\n## Environment Variables\n\nPlace in `.env` or environment:\n\n- `AHREFS_API_KEY` (required)\n- `AHREFS_BASE_URL` (default: Ahrefs API base or custom)\n- `AHREFS_TIMEOUT_S` (default: `10`)\n- `AHREFS_RATE_LIMIT_PER_MIN` (default: `120`)\n\nOptional (if supported in your `config.py`):\n- `AHREFS_AUTH_MODE` = `header` | `query` (default: `header`)\n\n## SDK Usage\n\n```python\nfrom backend.app.core.landing_page.ahrefs.client import AhrefsClient\n\nclient = AhrefsClient()\n\n# Site overview\nov = client.get_overview(target=\"example.com\")\nprint(ov)\n\n# Keywords Explorer\nkw = client.get_keywords_overview(query=\"coffee\")\nprint(kw)\n\n# Management: create project\nproj = client.create_project(name=\"Demo\", target=\"example.com\")\nprint(proj)\n\n# Public: crawler IPs\nips = client.get_crawler_ip_addresses()\nprint(ips)\n```\n\n## FastAPI Routes\n\nRouter is registered under prefix `/ahrefs`.\n\nExamples:\n- `GET /ahrefs/overview/overview?target=example.com`\n- `GET /ahrefs/serp/overview?query=best%20coffee`\n- `POST /ahrefs/batch-analysis` body: `{ \"items\": [\"a.com\", \"b.com\"] }`\n- `GET /ahrefs/subscription/limits-and-usage`\n- `GET /ahrefs/management/projects`\n- `POST /ahrefs/management/projects` body: `{ \"name\": \"Demo\", \"target\": \"example.com\" }`\n- `GET /ahrefs/public/crawler-ip-addresses`\n- `GET /ahrefs/public/crawler-ip-ranges`\n\nAll endpoints return a `GenericResponse` shape:\n\n```json\n{\n  \"ok\": true,\n  \"data\": { /* provider response payload */ }\n}\n```\n\n## Handlers Organization\n\nCategory-specific handlers under `api/`:\n\n- `backlinks_handlers.py`\n- `organic_handlers.py`\n- `paid_handlers.py`\n- `pages_handlers.py`\n- `outgoing_handlers.py`\n- `keywords_explorer_handlers.py`\n- `rank_tracker_handlers.py`\n- `overview_handlers.py`\n- `serp_handlers.py`\n- `batch_handlers.py`\n- `subscription_handlers.py`\n- `management_handlers.py`\n- `public_handlers.py`\n\nAll imported by `api/handlers.py`. Routes defined centrally in `api/routes.py`.\n\n## Errors\n\nCustom exceptions in `errors.py`:\n\n- `AhrefsAuthError` → 401\n- `AhrefsRateLimitError` → 429\n- `AhrefsAPIError` → 4xx/5xx mapping\n\n## Rate Limiting\n\nConfigured via `AHREFS_RATE_LIMIT_PER_MIN`. The SDK acquires a token per request and respects backpressure.\n\n## Testing\n\nTests live in `ahrefs/_tests/`.\n\nRun tests (PowerShell from repo root or tests dir):\n\n```bash\npytest backend/app/core/landing_page/ahrefs/_tests -q\n```\n\nCoverage is configured in `ahrefs/_tests/pytest.ini` to target this module.\n\nIncluded tests:\n- Client: smoke + representative endpoint calls with monkeypatched HTTP\n- API routes: FastAPI `TestClient` with DI override for the SDK\n- Errors: exception mapping (401/429/5xx)\n\nAdd-on ideas:\n- Config parsing and auth mode\n- Rate limiter timing and refill\n- More endpoint variations and edge cases\n\n## Security Notes\n\n- Never commit real API keys. Use `.env` and secret stores.\n- Prefer header-based auth unless query-mode is required.\n\n## Limitations\n\n- Response models are currently generic (`GenericResponse`). For stricter typing, add per-endpoint response models in `api/_responses.py` and update routes.\n\n## Changelog (high level)\n\n- Modular handlers per category and centralized routes\n- Implemented Site Explorer, Keywords Explorer, Rank Tracker, Overview, SERP, Batch, Subscription, Management, and Public Crawler endpoints\n- Added test suite with coverage config\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftechwithty%2Fahrefs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftechwithty%2Fahrefs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftechwithty%2Fahrefs/lists"}