{"id":16535637,"url":"https://github.com/hjsblogger/async-io-python","last_synced_at":"2026-04-17T22:31:08.984Z","repository":{"id":222706010,"uuid":"754120256","full_name":"hjsblogger/async-io-python","owner":"hjsblogger","description":"Demonstration of asyncio in Python using a string of different usecases (or test scenarios)","archived":false,"fork":false,"pushed_at":"2024-09-02T13:47:58.000Z","size":71,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-06T01:23:03.214Z","etag":null,"topics":["asyncio","automation","beautifulsoup","beautifulsoup4","python-asynchronous","python-asyncio","requests","selenium","selenium-python","web-scraping","webscraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hjsblogger.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-07T12:48:15.000Z","updated_at":"2024-09-02T13:48:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"e487d39d-983b-4c8b-8a8c-69c57d0deb16","html_url":"https://github.com/hjsblogger/async-io-python","commit_stats":null,"previous_names":["hjsblogger/async-io-python"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hjsblogger/async-io-python","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hjsblogger%2Fasync-io-python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hjsblogger%2Fasync-io-python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hjsblogger%2Fasync-io-python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hjsblogger%2Fasync-io-python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hjsblogger","download_url":"https://codeload.github.com/hjsblogger/async-io-python/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hjsblogger%2Fasync-io-python/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31948352,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-17T17:29:20.459Z","status":"ssl_error","status_checked_at":"2026-04-17T17:28:47.801Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","automation","beautifulsoup","beautifulsoup4","python-asynchronous","python-asyncio","requests","selenium","selenium-python","web-scraping","webscraping"],"created_at":"2024-10-11T18:28:24.918Z","updated_at":"2026-04-17T22:31:08.723Z","avatar_url":"https://github.com/hjsblogger.png","language":"Python","readme":"# Asyncio in Python\n\n\u003cimg width=\"1000\" height=\"400\" alt=\"Bulb\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/01bb4b8d-9243-4de3-9f02-bf4d19d99ac2\"\u003e\n\n\u003cdiv align=\"center\"\u003e\u003ca href=\"https://trunin.com/en/2021/07/python-call-async-from-sync/images/asyncio.jpeg\"\u003eImage Credit\u003c/a\u003e\u003c/div\u003e\n\u003cbr/\u003e\n\nIn this 'Asyncio with Python' repo, we have covered the following usecases:\n\n* Fetching Pokemon names using [Pokemon APIs](https://pokeapi.co/api/v2/pokemon/)\n* Fetching Weather information for certain cities in the US using [Openweather APIs](https://openweathermap.org/api)\n* Reading Automation Builds \u0026 Session information using [LambdaTest APIs](https://www.lambdatest.com/support/api-doc/)\n* Checking URL Health of links present on [LambdaTest Selenium Playground](https://lambdatest.com/selenium-playground)\n* Web Scraping of content on [LambdaTest E-Commerce Playground](https://ecommerce-playground.lambdatest.io/index.php?route=product/category\u0026path=57)\n\nThe scenarios mentioned above are executed using Synchronous \u0026 Asynchronous (or Async) modes in Python. The Async Programming is realized by leveraging the [asyncio](https://docs.python.org/3/library/asyncio.html) library in Python.  \n\n## Pre-requisites for test execution\n\n**Step 1**\n\nCreate a virtual environment by triggering the *virtualenv venv* command on the terminal\n\n```bash\nvirtualenv venv\n```\n\u003cimg width=\"1418\" alt=\"VirtualEnvironment\" src=\"https://github.com/hjsblogger/web-scraping-with-python/assets/1688653/89beb6af-549f-42ac-a063-e5f715018ef8\"\u003e\n\n**Step 2**\n\nNavigate the newly created virtual environment by triggering the *source venv/bin/activate* command on the terminal\n\n```bash\nsource venv/bin/activate\n```\n\nFollow steps(3) and (4) for using the LambdaTest Cloud Grid (particularly used for Web Scraping \u0026 URL Health Checking Scenarios):\n\n**Step 3**\n\nProcure the LambdaTest User Name and Access Key by navigating to [LambdaTest Account Page](https://accounts.lambdatest.com/security). You might need to create an an account on LambdaTest since it is used for running tests (or scraping) on the cloud Grid.\n\n\u003cimg width=\"1288\" alt=\"LambdaTestAccount\" src=\"https://github.com/hjsblogger/web-scraping-with-python/assets/1688653/9b40c9cb-93a1-4239-9fe5-99f33766a23a\"\u003e\n\n**Step 4**\n\nAdd the LambdaTest User Name and Access Key in the *.env* (or *Makefile*)that is located in the parent directory. Once done, save the Makefile.\n\n\u003cimg width=\"600\" height=\"300\" alt=\"LambdaTestEnv-Change\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/ffe23c56-5b53-4d6f-8cfa-85c82d725f99\"\u003e\n\n\u003cimg width=\"600\" height=\"300\" alt=\"LambdaTestAccount\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/a3105b0c-4515-448b-ace3-77b25a9bf2c1\"\u003e\n\n**Step 5**\n\nFor realizing the *Weather Information* scenario, you need to register on [OpenWeather](https://openweathermap.org/). Once done, set the environment variable *OPEN_WEATHER_API* in .env. The API keys can be located from [OpenWeather API Page](https://home.openweathermap.org/api_keys)\n\n\u003cimg width=\"1000\" alt=\"OpenWeather-API-2\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/7d2ebb75-2d72-4dc6-9fb5-bf9cf2a7c96f\"\u003e\n\n\u003cimg width=\"600\" height=\"300\" alt=\"OpenWeather-API-1\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/e240c220-bb62-491b-a423-f61e34183ec1\"\u003e\n\n## Dependency/Package Installation\n\nRun the *make install* command on the terminal to install the desired packages (or dependencies) - Pytest, Selenium, Beautiful Soup, etc.\n\n```bash\nmake install\n```\n\u003cimg width=\"1295\" alt=\"Makefile-1\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/9dd1d433-8994-47ec-b504-fe8c8810979f\"\u003e\n\n\u003cimg width=\"1412\" alt=\"Makefile-2\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/062f66c6-2bbd-44dc-bfad-807c7e006d55\"\u003e\n\nFor benchmarking (over a certain number of runs), we have used [hyperfine](https://github.com/sharkdp/hyperfine), a command-line benchmarking tool. Installation of hyperfine on macOS is done by triggering ```brew install hyperfine``` on the terminal.\n\n\u003cimg width=\"1405\" alt=\"Hyperfine-1\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/053b9437-5065-4176-8e91-08e81a2d5b96\"\u003e\n\nWith this, all the dependencies and environment variables are set. It's time for some action !!\n\n## Execution\n\nFollow the below mentioned steps to benchmark the performance of sync and async code (via the hyperfine tool):\n\n**Step 1**\n\nFor the URL health checking scenario, the Chrome browser is invoked in the non-headless mode. It is recommended to install Chrome on your machine before you proceed to Step(4)\n\n**Step 2**\n\nTrigger the command *make clean* to clean the remove _pycache_ folder(s) and .pyc files. It also cleans .DS_Store folder from the project.\n\n\u003cimg width=\"506\" alt=\"MakeClean\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/5b1bbb77-1a79-4586-940f-c07f9a0cbb69\"\u003e\n\n**Step 3**\n\nTrigger the *make* command on the terminal to realize the usecases in the sync \u0026 async mode.\n\n### Fetching Pokemon names using Pokemon APIs\n\nTrigger the command *make fetch-pokemon-names* to fetch the Pokemon names using [Pokemon APIs](https://pokeapi.co/api/v2/pokemon/) and running code in sync \u0026 async mode (using Asyncio in Python).\n\n\u003cimg width=\"1106\" alt=\"1_Pokemon_Execution\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/98c543cc-1474-4c4d-a7ba-88079730d168\"\u003e\n\n### \u003cb\u003eResult: Asyncio - 5.77+ seconds faster than sync execution\u003c/b\u003e\n\n### Fetching Weather information for certain cities in the US using Openweather APIs\n\nIn this scenario, the city names from [Page-1](https://www.latlong.net/category/cities-236-15-1.html) thru' [Page-15](https://www.latlong.net/category/cities-236-15-13.html) are first scraped using BeautifulSoup.\n\n\u003cimg width=\"1422\" alt=\"WeatherScenario\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/683963c7-7dbf-4b07-b20f-c3dabb3427cf\"\u003e\n\nNow that we have the city names, let's fetch the weather of those cities by providing Latitude \u0026 Longitude to the [OpenWeather API](https://api.openweathermap.org/data/2.5/weather?lat=\u003cLATITUDE\u003e\u0026lon=\u003cLONGITUDE\u003e\u0026appid=\u003cOPEN_WEATHER_API\u003e)\n\nTrigger the command *make fetch-sync-weather-info* to fetch the Pokemon names using [Pokemon APIs](https://pokeapi.co/api/v2/pokemon/) and running code in sync mode.\n\n\u003cimg width=\"1099\" alt=\"2 1_Weather_Info_Sync_Execution\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/7f6f1545-8d45-4837-98f4-cdd5b1670250\"\u003e\n\n\u003cimg width=\"1099\" alt=\"2 2_Weather_Info_Sync_Execution\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/99275e78-9fc7-4619-b7c5-25fea0911da1\"\u003e\n\n\u003cimg width=\"1099\" alt=\"2 3_Weather_Info_Sync_Execution\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/9603f535-45fe-4a2f-8f6f-e02f6a32c0ea\"\u003e\n\nTrigger the command *make fetch-async-weather-info* to fetch the Pokemon names using [Pokemon APIs](https://pokeapi.co/api/v2/pokemon/) and running code in sync \u0026 async mode (using Asyncio in Python).\n\n\u003cimg width=\"1097\" alt=\"2 4_Weather_Info_ASync_Execution\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/a78b8bd6-dd40-4bc3-a4c4-0a19014201ed\"\u003e\n\n\u003cimg width=\"1097\" alt=\"2 5_Weather_Info_ASync_Execution\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/5a765c29-839c-4551-ac8b-26ec90b64ce1\"\u003e\n\n\u003cimg width=\"1097\" alt=\"2 6_Weather_Info_ASync_Execution\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/b2dbfdac-78c7-4af7-ae41-53e046876640\"\u003e\n\n### \u003cb\u003eResult: Asyncio - 318.53+ seconds faster than sync execution\u003c/b\u003e\n\n### Checking URL health\n\nIn this scenario, the hyperlinks on [LambdaTest Selenium Playground](https://lambdatest.com/selenium-playground) are first scraped using BeautifulSoup. Once the links are available, their health is checked using the response code via the [requests](https://pypi.org/project/requests/) library\n\n\u003cimg width=\"1427\" alt=\"URL-Health-Checking\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/b1be601c-eb7b-4314-b898-b4ddbc8b31e9\"\u003e\n\nTrigger the command *make check-url-health* to check the health (i.e. response code) of the links present on the LambdaTest Selenium Playground.\n\n\u003cimg width=\"1110\" alt=\"4_Check_URL_Health_Execution\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/bd4c7c89-38fa-4e80-99c9-411328bcd151\"\u003e\n\n### \u003cb\u003eResult: Asyncio - 1.41+ seconds faster than sync execution\u003c/b\u003e\n\n### Web Scraping\n\nIn this scenario, the product details from [Page-1](https://ecommerce-playground.lambdatest.io/index.php?route=product/category\u0026path=57) thru' [Page-5](https://ecommerce-playground.lambdatest.io/index.php?route=product/category\u0026path=57\u0026page=5) are scraped using the BeautifulSoup library.\n\nThe implementation using the sync programming is similar to the Web Scraping scenario that covered in [Web Scraping with Python](https://github.com/hjsblogger/web-scraping-with-python/blob/main/tests/beautiful-soup/test_ecommerce_scraping.py) repo.\n\nTrigger the command *make perform-web-scraping* to check scrap the product details on the test page mentioned earlier.\n\n\u003cimg width=\"1110\" alt=\"5_Web_Scraping_Execution\" src=\"https://github.com/hjsblogger/async-io-python/assets/1688653/2a40bbc2-7b50-4be3-96e4-357971f749e3\"\u003e\n\n### \u003cb\u003eResult: Asyncio - 2.93+ seconds faster than sync execution\u003c/b\u003e\n\n## Have feedback or need assistance?\nFeel free to fork the repo and contribute to make it better! Email to [himanshu[dot]sheth[at]gmail[dot]com](mailto:himanshu.sheth@gmail.com) for any queries or ping me on the following social media sites:\n\n\u003cb\u003eLinkedIn\u003c/b\u003e: [@hjsblogger](https://linkedin.com/in/hjsblogger)\u003cbr/\u003e\n\u003cb\u003eTwitter\u003c/b\u003e: [@hjsblogger](https://www.twitter.com/hjsblogger)","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhjsblogger%2Fasync-io-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhjsblogger%2Fasync-io-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhjsblogger%2Fasync-io-python/lists"}