{"id":22371310,"url":"https://github.com/itielshwartz/asyncio-hn","last_synced_at":"2025-07-27T14:08:24.631Z","repository":{"id":46084660,"uuid":"80563037","full_name":"itielshwartz/asyncio-hn","owner":"itielshwartz","description":"Python (asyncio) wrapper for hackernews api","archived":false,"fork":false,"pushed_at":"2022-12-26T20:24:44.000Z","size":20,"stargazers_count":28,"open_issues_count":5,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-09T17:15:18.447Z","etag":null,"topics":["api","asyncio","hacker-news","hn","python","python36","scraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/itielshwartz.png","metadata":{"files":{"readme":"README","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-01-31T21:15:34.000Z","updated_at":"2023-02-20T15:21:28.000Z","dependencies_parsed_at":"2023-01-31T01:15:14.748Z","dependency_job_id":null,"html_url":"https://github.com/itielshwartz/asyncio-hn","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/itielshwartz/asyncio-hn","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itielshwartz%2Fasyncio-hn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itielshwartz%2Fasyncio-hn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itielshwartz%2Fasyncio-hn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itielshwartz%2Fasyncio-hn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/itielshwartz","download_url":"https://codeload.github.com/itielshwartz/asyncio-hn/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itielshwartz%2Fasyncio-hn/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267368932,"owners_count":24076093,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-27T02:00:11.917Z","response_time":82,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","asyncio","hacker-news","hn","python","python36","scraping"],"created_at":"2024-12-04T20:18:50.200Z","updated_at":"2025-07-27T14:08:24.606Z","avatar_url":"https://github.com/itielshwartz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"asyncio-hn\n==========\n\nA simple asyncio wrapper to download\n`hacker-news \u003chttps://news.ycombinator.com/\u003e`__ with speed and ease.\n\nThe package supports all endpoints of the official API : `hacker-news\nAPI \u003chttps://github.com/HackerNews/API\u003e`__\n\nInstallation\n------------\n\n.. code:: shell\n\n    pip install asyncio-hn\n\nUsage\n-----\n\n.. code:: python\n\n    import asyncio\n    from asyncio_hn import ClientHN\n\n    async def main(loop):\n        # We init the client - extension of aiohttp.ClientSession\n        async with ClientHN(loop=loop) as hn:\n            # Up to 500 top and top stories (only ids)\n            hn_new_stories = await hn.top_stories()\n            # Download top 10 story data\n            top_posts = await hn.items(hn_new_stories[:10])\n            # Download the user data for each story\n            users = await hn.users([post.get(\"by\") for post in top_posts])\n\n\n    if __name__ == '__main__':\n        loop = asyncio.get_event_loop()\n        loop.run_until_complete(main(loop))\n\nAdvance usage\n~~~~~~~~~~~~~\n\nUsing this config you can reach 1000+ request/sec.\n\n.. code:: python\n    import aiohttp\n    from asyncio_hn import ClientHN\n    \n    N = 1_000_000\n\n    async def advance_run(loop):\n        # We init the client - extension of aiohttp.ClientSession\n        conn = aiohttp.TCPConnector(limit=1000, loop=loop)\n        async with ClientHN(loop=loop, queue_size=1000, connector=conn, progress_bar=True, debug=True) as hn:\n            # Download the last 1,000,000 stories\n            hn_new_stories = await hn.last_n_items(n=N)\n\nOutput example:\n---------------\n\nItem:\n\n.. code:: python\n\n    item = {'by': 'amzans', 'descendants': 25, 'id': 13566716,\n                    'kids': [13567061, 13567631, 13567027, 13567055, 13566798, 13567473], 'score': 171, 'time': 1486210548,\n                    'title': 'Network programming with Go (2012)', 'type': 'story',\n                    'url': 'https://jannewmarch.gitbooks.io/network-programming-with-go-golang-/content/'},\n                   {'by': 'r3bl', 'descendants': 1, 'id': 13567940, 'kids': [13568249], 'score': 24, 'time': 1486230224,\n                    'title': 'YouTube removes hundreds of the best climate science videos from the Internet',\n                    'type': 'story',\n                    'url': 'http://climatestate.com/2017/02/03/youtube-removes-hundreds-of-the-best-climate-science-videos-from-the-internet/'}\n\nUser:\n\n.. code:: python\n\n    user = {'created': 1470758993, 'id': 'amzans', 'karma': 174,\n            'submitted': [13567884, 13566716, 13566699, 13558456, 13539270, 13539151, 13514498, 13418469, 13417725,\n                          13416562, 13416097, 13416034, 13415954, 13415894, 13395310, 13394996, 13392554, 12418804,\n                          12418361, 12413958, 12411992, 12411732, 12411546, 12262383, 12255593]}\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fitielshwartz%2Fasyncio-hn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fitielshwartz%2Fasyncio-hn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fitielshwartz%2Fasyncio-hn/lists"}