{"id":18710640,"url":"https://github.com/apify/apify-sdk-python","last_synced_at":"2026-02-25T18:07:58.050Z","repository":{"id":65794862,"uuid":"573436939","full_name":"apify/apify-sdk-python","owner":"apify","description":"The Apify SDK for Python is the official library for creating Apify Actors in Python. It provides useful features like actor lifecycle management, local storage emulation, and actor event handling.","archived":false,"fork":false,"pushed_at":"2025-04-01T07:28:29.000Z","size":6951,"stargazers_count":128,"open_issues_count":12,"forks_count":11,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-04-01T07:48:00.357Z","etag":null,"topics":["apify","automation","python","scraping","sdk"],"latest_commit_sha":null,"homepage":"https://docs.apify.com/sdk/python","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apify.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-02T13:12:31.000Z","updated_at":"2025-04-01T07:28:32.000Z","dependencies_parsed_at":"2024-03-11T14:54:16.304Z","dependency_job_id":"085472cf-bbc5-44a7-9b5e-6155ff5b6c12","html_url":"https://github.com/apify/apify-sdk-python","commit_stats":{"total_commits":293,"total_committers":19,"mean_commits":"15.421052631578947","dds":0.7133105802047781,"last_synced_commit":"89941af26c5d5cc6e2a3c2ef2c609009817f39df"},"previous_names":[],"tags_count":203,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apify%2Fapify-sdk-python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apify%2Fapify-sdk-python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apify%2Fapify-sdk-python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apify%2Fapify-sdk-python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apify","download_url":"https://codeload.github.com/apify/apify-sdk-python/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247792888,"owners_count":20996900,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apify","automation","python","scraping","sdk"],"created_at":"2024-11-07T12:35:07.603Z","updated_at":"2026-02-25T18:07:58.042Z","avatar_url":"https://github.com/apify.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=center\u003eApify SDK for Python\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://badge.fury.io/py/apify\" rel=\"nofollow\"\u003e\u003cimg src=\"https://badge.fury.io/py/apify.svg\" alt=\"PyPI package version\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/apify/\" rel=\"nofollow\"\u003e\u003cimg src=\"https://img.shields.io/pypi/dm/apify\" alt=\"PyPI package downloads\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://codecov.io/gh/apify/apify-sdk-python\"\u003e\u003cimg src=\"https://codecov.io/gh/apify/apify-sdk-python/graph/badge.svg?token=Y6JBIZQFT6\" alt=\"Codecov report\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/apify/\" rel=\"nofollow\"\u003e\u003cimg src=\"https://img.shields.io/pypi/pyversions/apify\" alt=\"PyPI Python version\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://discord.gg/jyEM2PRvMU\" rel=\"nofollow\"\u003e\u003cimg src=\"https://img.shields.io/discord/801163717915574323?label=discord\" alt=\"Chat on Discord\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\nThe Apify SDK for Python is the official library to create [Apify Actors](https://docs.apify.com/platform/actors)\nin Python. It provides useful features like Actor lifecycle management, local storage emulation, and Actor\nevent handling.\n\nIf you just need to access the [Apify API](https://docs.apify.com/api/v2) from your Python applications,\ncheck out the [Apify Client for Python](https://docs.apify.com/api/client/python) instead.\n\n## Installation\n\nThe Apify SDK for Python is available on PyPI as the `apify` package.\nFor default installation, using Pip, run the following:\n\n```bash\npip install apify\n```\n\nFor users interested in integrating Apify with Scrapy, we provide a package extra called `scrapy`.\nTo install Apify with the `scrapy` extra, use the following command:\n\n```bash\npip install apify[scrapy]\n```\n\n## Documentation\n\nFor usage instructions, check the documentation on [Apify Docs](https://docs.apify.com/sdk/python/).\n\n## Examples\n\nBelow are few examples demonstrating how to use the Apify SDK with some web scraping-related libraries.\n\n### Apify SDK with HTTPX and BeautifulSoup\n\nThis example illustrates how to integrate the Apify SDK with [HTTPX](https://www.python-httpx.org/) and [BeautifulSoup](https://pypi.org/project/beautifulsoup4/) to scrape data from web pages.\n\n```python\nfrom bs4 import BeautifulSoup\nfrom httpx import AsyncClient\n\nfrom apify import Actor\n\n\nasync def main() -\u003e None:\n    async with Actor:\n        # Retrieve the Actor input, and use default values if not provided.\n        actor_input = await Actor.get_input() or {}\n        start_urls = actor_input.get('start_urls', [{'url': 'https://apify.com'}])\n\n        # Open the default request queue for handling URLs to be processed.\n        request_queue = await Actor.open_request_queue()\n\n        # Enqueue the start URLs.\n        for start_url in start_urls:\n            url = start_url.get('url')\n            await request_queue.add_request(url)\n\n        # Process the URLs from the request queue.\n        while request := await request_queue.fetch_next_request():\n            Actor.log.info(f'Scraping {request.url} ...')\n\n            # Fetch the HTTP response from the specified URL using HTTPX.\n            async with AsyncClient() as client:\n                response = await client.get(request.url)\n\n            # Parse the HTML content using Beautiful Soup.\n            soup = BeautifulSoup(response.content, 'html.parser')\n\n            # Extract the desired data.\n            data = {\n                'url': actor_input['url'],\n                'title': soup.title.string,\n                'h1s': [h1.text for h1 in soup.find_all('h1')],\n                'h2s': [h2.text for h2 in soup.find_all('h2')],\n                'h3s': [h3.text for h3 in soup.find_all('h3')],\n            }\n\n            # Store the extracted data to the default dataset.\n            await Actor.push_data(data)\n```\n\n### Apify SDK with PlaywrightCrawler from Crawlee\n\nThis example demonstrates how to use the Apify SDK alongside `PlaywrightCrawler` from [Crawlee](https://crawlee.dev/python) to perform web scraping.\n\n```python\nfrom crawlee.crawlers import PlaywrightCrawler, PlaywrightCrawlingContext\n\nfrom apify import Actor\n\n\nasync def main() -\u003e None:\n    async with Actor:\n        # Retrieve the Actor input, and use default values if not provided.\n        actor_input = await Actor.get_input() or {}\n        start_urls = [url.get('url') for url in actor_input.get('start_urls', [{'url': 'https://apify.com'}])]\n\n        # Exit if no start URLs are provided.\n        if not start_urls:\n            Actor.log.info('No start URLs specified in Actor input, exiting...')\n            await Actor.exit()\n\n        # Create a crawler.\n        crawler = PlaywrightCrawler(\n            # Limit the crawl to max requests. Remove or increase it for crawling all links.\n            max_requests_per_crawl=50,\n            headless=True,\n        )\n\n        # Define a request handler, which will be called for every request.\n        @crawler.router.default_handler\n        async def request_handler(context: PlaywrightCrawlingContext) -\u003e None:\n            url = context.request.url\n            Actor.log.info(f'Scraping {url}...')\n\n            # Extract the desired data.\n            data = {\n                'url': context.request.url,\n                'title': await context.page.title(),\n                'h1s': [await h1.text_content() for h1 in await context.page.locator('h1').all()],\n                'h2s': [await h2.text_content() for h2 in await context.page.locator('h2').all()],\n                'h3s': [await h3.text_content() for h3 in await context.page.locator('h3').all()],\n            }\n\n            # Store the extracted data to the default dataset.\n            await context.push_data(data)\n\n            # Enqueue additional links found on the current page.\n            await context.enqueue_links()\n\n        # Run the crawler with the starting URLs.\n        await crawler.run(start_urls)\n```\n\n## What are Actors?\n\nActors are serverless cloud programs that can do almost anything a human can do in a web browser.\nThey can do anything from small tasks such as filling in forms or unsubscribing from online services,\nall the way up to scraping and processing vast numbers of web pages.\n\nThey can be run either locally, or on the [Apify platform](https://docs.apify.com/platform/),\nwhere you can run them at scale, monitor them, schedule them, or publish and monetize them.\n\nIf you're new to Apify, learn [what is Apify](https://docs.apify.com/platform/about)\nin the Apify platform documentation.\n\n## Creating Actors\n\nTo create and run Actors through Apify Console,\nsee the [Console documentation](https://docs.apify.com/academy/getting-started/creating-actors#choose-your-template).\n\nTo create and run Python Actors locally, check the documentation for\n[how to create and run Python Actors locally](https://docs.apify.com/sdk/python/docs/quick-start).\n\n## Guides\n\nTo see how you can use the Apify SDK with other popular libraries used for web scraping,\ncheck out our guides for using\n[Requests and HTTPX](https://docs.apify.com/sdk/python/docs/guides/requests-and-httpx),\n[Beautiful Soup](https://docs.apify.com/sdk/python/docs/guides/beautiful-soup),\n[Playwright](https://docs.apify.com/sdk/python/docs/guides/playwright),\n[Selenium](https://docs.apify.com/sdk/python/docs/guides/selenium),\nor [Scrapy](https://docs.apify.com/sdk/python/docs/guides/scrapy).\n\n## Usage concepts\n\nTo learn more about the features of the Apify SDK and how to use them,\ncheck out the Usage Concepts section in the sidebar,\nparticularly the guides for the [Actor lifecycle](https://docs.apify.com/sdk/python/docs/concepts/actor-lifecycle),\n[working with storages](https://docs.apify.com/sdk/python/docs/concepts/storages),\n[handling Actor events](https://docs.apify.com/sdk/python/docs/concepts/actor-events)\nor [how to use proxies](https://docs.apify.com/sdk/python/docs/concepts/proxy-management).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapify%2Fapify-sdk-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapify%2Fapify-sdk-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapify%2Fapify-sdk-python/lists"}