{"id":13441125,"url":"https://github.com/psf/requests-html","last_synced_at":"2025-05-13T19:10:31.244Z","repository":{"id":37493439,"uuid":"122775780","full_name":"psf/requests-html","owner":"psf","description":"Pythonic HTML Parsing for Humans™","archived":false,"fork":false,"pushed_at":"2024-04-16T18:50:38.000Z","size":2981,"stargazers_count":13809,"open_issues_count":234,"forks_count":988,"subscribers_count":268,"default_branch":"master","last_synced_at":"2025-05-09T00:10:07.284Z","etag":null,"topics":["beautifulsoup","css-selectors","html","http","kennethreitz","lxml","pyquery","python","requests","scraping"],"latest_commit_sha":null,"homepage":"http://html.python-requests.org","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/psf.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-02-24T20:11:02.000Z","updated_at":"2025-05-07T20:56:06.000Z","dependencies_parsed_at":"2024-04-28T03:54:35.730Z","dependency_job_id":"e1b481f8-f69e-4b67-97e9-00058415e483","html_url":"https://github.com/psf/requests-html","commit_stats":{"total_commits":367,"total_committers":64,"mean_commits":5.734375,"dds":0.3215258855585831,"last_synced_commit":"87a047d4adb37125004ab52d763e6294db4d9b6f"},"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psf%2Frequests-html","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psf%2Frequests-html/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psf%2Frequests-html/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psf%2Frequests-html/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/psf","download_url":"https://codeload.github.com/psf/requests-html/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254010823,"owners_count":21998993,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beautifulsoup","css-selectors","html","http","kennethreitz","lxml","pyquery","python","requests","scraping"],"created_at":"2024-07-31T03:01:30.224Z","updated_at":"2025-05-13T19:10:31.224Z","avatar_url":"https://github.com/psf.png","language":"Python","readme":"Requests-HTML: HTML Parsing for Humans™\n=======================================\n\n.. image:: https://farm5.staticflickr.com/4695/39152770914_a3ab8af40d_k_d.jpg\n\n.. image:: https://travis-ci.com/psf/requests-html.svg?branch=master\n    :target: https://travis-ci.com/psf/requests-html\n\nThis library intends to make parsing HTML (e.g. scraping the web) as\nsimple and intuitive as possible.\n\nWhen using this library you automatically get:\n\n- **Full JavaScript support**! (Using Chromium, thanks to pyppeteer)\n- *CSS Selectors* (a.k.a jQuery-style, thanks to PyQuery).\n- *XPath Selectors*, for the faint of heart.\n- Mocked user-agent (like a real web browser).\n- Automatic following of redirects.\n- Connection–pooling and cookie persistence.\n- The Requests experience you know and love, with magical parsing abilities.\n- **Async Support**\n\n.. Other nice features include:\n\n    - Markdown export of pages and elements.\n\n\nTutorial \u0026 Usage\n================\n\nMake a GET request to 'python.org', using Requests:\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e from requests_html import HTMLSession\n    \u003e\u003e\u003e session = HTMLSession()\n    \u003e\u003e\u003e r = session.get('https://python.org/')\n\nTry async and get some sites at the same time:\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e from requests_html import AsyncHTMLSession\n    \u003e\u003e\u003e asession = AsyncHTMLSession()\n    \u003e\u003e\u003e async def get_pythonorg():\n    ...     r = await asession.get('https://python.org/')\n    ...     return r\n    ...\n    \u003e\u003e\u003e async def get_reddit():\n    ...    r = await asession.get('https://reddit.com/')\n    ...    return r\n    ...\n    \u003e\u003e\u003e async def get_google():\n    ...    r = await asession.get('https://google.com/')\n    ...    return r\n    ...\n    \u003e\u003e\u003e results = asession.run(get_pythonorg, get_reddit, get_google)\n    \u003e\u003e\u003e results # check the requests all returned a 200 (success) code\n    [\u003cResponse [200]\u003e, \u003cResponse [200]\u003e, \u003cResponse [200]\u003e]\n    \u003e\u003e\u003e # Each item in the results list is a response object and can be interacted with as such\n    \u003e\u003e\u003e for result in results: \n    ...     print(result.html.url)\n    ... \n    https://www.python.org/\n    https://www.google.com/\n    https://www.reddit.com/\n\nNote that the order of the objects in the results list represents the order they were returned in, not the order that the coroutines are passed to the ``run`` method, which is shown in the example by the order being different. \n\nGrab a list of all links on the page, as–is (anchors excluded):\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e r.html.links\n    {'//docs.python.org/3/tutorial/', '/about/apps/', 'https://github.com/python/pythondotorg/issues', '/accounts/login/', '/dev/peps/', '/about/legal/', '//docs.python.org/3/tutorial/introduction.html#lists', '/download/alternatives', 'http://feedproxy.google.com/~r/PythonInsider/~3/kihd2DW98YY/python-370a4-is-available-for-testing.html', '/download/other/', '/downloads/windows/', 'https://mail.python.org/mailman/listinfo/python-dev', '/doc/av', 'https://devguide.python.org/', '/about/success/#engineering', 'https://wiki.python.org/moin/PythonEventsCalendar#Submitting_an_Event', 'https://www.openstack.org', '/about/gettingstarted/', 'http://feedproxy.google.com/~r/PythonInsider/~3/AMoBel8b8Mc/python-3.html', '/success-stories/industrial-light-magic-runs-python/', 'http://docs.python.org/3/tutorial/introduction.html#using-python-as-a-calculator', '/', 'http://pyfound.blogspot.com/', '/events/python-events/past/', '/downloads/release/python-2714/', 'https://wiki.python.org/moin/PythonBooks', 'http://plus.google.com/+Python', 'https://wiki.python.org/moin/', 'https://status.python.org/', '/community/workshops/', '/community/lists/', 'http://buildbot.net/', '/community/awards', 'http://twitter.com/ThePSF', 'https://docs.python.org/3/license.html', '/psf/donations/', 'http://wiki.python.org/moin/Languages', '/dev/', '/events/python-user-group/', 'https://wiki.qt.io/PySide', '/community/sigs/', 'https://wiki.gnome.org/Projects/PyGObject', 'http://www.ansible.com', 'http://www.saltstack.com', 'http://planetpython.org/', '/events/python-events', '/about/help/', '/events/python-user-group/past/', '/about/success/', '/psf-landing/', '/about/apps', '/about/', 'http://www.wxpython.org/', '/events/python-user-group/665/', 'https://www.python.org/psf/codeofconduct/', '/dev/peps/peps.rss', '/downloads/source/', '/psf/sponsorship/sponsors/', 'http://bottlepy.org', 'http://roundup.sourceforge.net/', 'http://pandas.pydata.org/', 'http://brochure.getpython.info/', 'https://bugs.python.org/', '/community/merchandise/', 'http://tornadoweb.org', '/events/python-user-group/650/', 'http://flask.pocoo.org/', '/downloads/release/python-364/', '/events/python-user-group/660/', '/events/python-user-group/638/', '/psf/', '/doc/', 'http://blog.python.org', '/events/python-events/604/', '/about/success/#government', 'http://python.org/dev/peps/', 'https://docs.python.org', 'http://feedproxy.google.com/~r/PythonInsider/~3/zVC80sq9s00/python-364-is-now-available.html', '/users/membership/', '/about/success/#arts', 'https://wiki.python.org/moin/Python2orPython3', '/downloads/', '/jobs/', 'http://trac.edgewall.org/', 'http://feedproxy.google.com/~r/PythonInsider/~3/wh73_1A-N7Q/python-355rc1-and-python-348rc1-are-now.html', '/privacy/', 'https://pypi.python.org/', 'http://www.riverbankcomputing.co.uk/software/pyqt/intro', 'http://www.scipy.org', '/community/forums/', '/about/success/#scientific', '/about/success/#software-development', '/shell/', '/accounts/signup/', 'http://www.facebook.com/pythonlang?fref=ts', '/community/', 'https://kivy.org/', '/about/quotes/', 'http://www.web2py.com/', '/community/logos/', '/community/diversity/', '/events/calendars/', 'https://wiki.python.org/moin/BeginnersGuide', '/success-stories/', '/doc/essays/', '/dev/core-mentorship/', 'http://ipython.org', '/events/', '//docs.python.org/3/tutorial/controlflow.html', '/about/success/#education', '/blogs/', '/community/irc/', 'http://pycon.blogspot.com/', '//jobs.python.org', 'http://www.pylonsproject.org/', 'http://www.djangoproject.com/', '/downloads/mac-osx/', '/about/success/#business', 'http://feedproxy.google.com/~r/PythonInsider/~3/x_c9D0S-4C4/python-370b1-is-now-available-for.html', 'http://wiki.python.org/moin/TkInter', 'https://docs.python.org/faq/', '//docs.python.org/3/tutorial/controlflow.html#defining-functions'}\n\nGrab a list of all links on the page, in absolute form (anchors excluded):\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e r.html.absolute_links\n    {'https://github.com/python/pythondotorg/issues', 'https://docs.python.org/3/tutorial/', 'https://www.python.org/about/success/', 'http://feedproxy.google.com/~r/PythonInsider/~3/kihd2DW98YY/python-370a4-is-available-for-testing.html', 'https://www.python.org/dev/peps/', 'https://mail.python.org/mailman/listinfo/python-dev', 'https://www.python.org/doc/', 'https://www.python.org/', 'https://www.python.org/about/', 'https://www.python.org/events/python-events/past/', 'https://devguide.python.org/', 'https://wiki.python.org/moin/PythonEventsCalendar#Submitting_an_Event', 'https://www.openstack.org', 'http://feedproxy.google.com/~r/PythonInsider/~3/AMoBel8b8Mc/python-3.html', 'https://docs.python.org/3/tutorial/introduction.html#lists', 'http://docs.python.org/3/tutorial/introduction.html#using-python-as-a-calculator', 'http://pyfound.blogspot.com/', 'https://wiki.python.org/moin/PythonBooks', 'http://plus.google.com/+Python', 'https://wiki.python.org/moin/', 'https://www.python.org/events/python-events', 'https://status.python.org/', 'https://www.python.org/about/apps', 'https://www.python.org/downloads/release/python-2714/', 'https://www.python.org/psf/donations/', 'http://buildbot.net/', 'http://twitter.com/ThePSF', 'https://docs.python.org/3/license.html', 'http://wiki.python.org/moin/Languages', 'https://docs.python.org/faq/', 'https://jobs.python.org', 'https://www.python.org/about/success/#software-development', 'https://www.python.org/about/success/#education', 'https://www.python.org/community/logos/', 'https://www.python.org/doc/av', 'https://wiki.qt.io/PySide', 'https://www.python.org/events/python-user-group/660/', 'https://wiki.gnome.org/Projects/PyGObject', 'http://www.ansible.com', 'http://www.saltstack.com', 'https://www.python.org/dev/peps/peps.rss', 'http://planetpython.org/', 'https://www.python.org/events/python-user-group/past/', 'https://docs.python.org/3/tutorial/controlflow.html#defining-functions', 'https://www.python.org/community/diversity/', 'https://docs.python.org/3/tutorial/controlflow.html', 'https://www.python.org/community/awards', 'https://www.python.org/events/python-user-group/638/', 'https://www.python.org/about/legal/', 'https://www.python.org/dev/', 'https://www.python.org/download/alternatives', 'https://www.python.org/downloads/', 'https://www.python.org/community/lists/', 'http://www.wxpython.org/', 'https://www.python.org/about/success/#government', 'https://www.python.org/psf/', 'https://www.python.org/psf/codeofconduct/', 'http://bottlepy.org', 'http://roundup.sourceforge.net/', 'http://pandas.pydata.org/', 'http://brochure.getpython.info/', 'https://www.python.org/downloads/source/', 'https://bugs.python.org/', 'https://www.python.org/downloads/mac-osx/', 'https://www.python.org/about/help/', 'http://tornadoweb.org', 'http://flask.pocoo.org/', 'https://www.python.org/users/membership/', 'http://blog.python.org', 'https://www.python.org/privacy/', 'https://www.python.org/about/gettingstarted/', 'http://python.org/dev/peps/', 'https://www.python.org/about/apps/', 'https://docs.python.org', 'https://www.python.org/success-stories/', 'https://www.python.org/community/forums/', 'http://feedproxy.google.com/~r/PythonInsider/~3/zVC80sq9s00/python-364-is-now-available.html', 'https://www.python.org/community/merchandise/', 'https://www.python.org/about/success/#arts', 'https://wiki.python.org/moin/Python2orPython3', 'http://trac.edgewall.org/', 'http://feedproxy.google.com/~r/PythonInsider/~3/wh73_1A-N7Q/python-355rc1-and-python-348rc1-are-now.html', 'https://pypi.python.org/', 'https://www.python.org/events/python-user-group/650/', 'http://www.riverbankcomputing.co.uk/software/pyqt/intro', 'https://www.python.org/about/quotes/', 'https://www.python.org/downloads/windows/', 'https://www.python.org/events/calendars/', 'http://www.scipy.org', 'https://www.python.org/community/workshops/', 'https://www.python.org/blogs/', 'https://www.python.org/accounts/signup/', 'https://www.python.org/events/', 'https://kivy.org/', 'http://www.facebook.com/pythonlang?fref=ts', 'http://www.web2py.com/', 'https://www.python.org/psf/sponsorship/sponsors/', 'https://www.python.org/community/', 'https://www.python.org/download/other/', 'https://www.python.org/psf-landing/', 'https://www.python.org/events/python-user-group/665/', 'https://wiki.python.org/moin/BeginnersGuide', 'https://www.python.org/accounts/login/', 'https://www.python.org/downloads/release/python-364/', 'https://www.python.org/dev/core-mentorship/', 'https://www.python.org/about/success/#business', 'https://www.python.org/community/sigs/', 'https://www.python.org/events/python-user-group/', 'http://ipython.org', 'https://www.python.org/shell/', 'https://www.python.org/community/irc/', 'https://www.python.org/about/success/#engineering', 'http://www.pylonsproject.org/', 'http://pycon.blogspot.com/', 'https://www.python.org/about/success/#scientific', 'https://www.python.org/doc/essays/', 'http://www.djangoproject.com/', 'https://www.python.org/success-stories/industrial-light-magic-runs-python/', 'http://feedproxy.google.com/~r/PythonInsider/~3/x_c9D0S-4C4/python-370b1-is-now-available-for.html', 'http://wiki.python.org/moin/TkInter', 'https://www.python.org/jobs/', 'https://www.python.org/events/python-events/604/'}\n\nSelect an element with a CSS Selector:\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e about = r.html.find('#about', first=True)\n\nGrab an element's text contents:\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e print(about.text)\n    About\n    Applications\n    Quotes\n    Getting Started\n    Help\n    Python Brochure\n\nIntrospect an Element's attributes:\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e about.attrs\n    {'id': 'about', 'class': ('tier-1', 'element-1'), 'aria-haspopup': 'true'}\n\nRender out an Element's HTML:\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e about.html\n    '\u003cli aria-haspopup=\"true\" class=\"tier-1 element-1 \" id=\"about\"\u003e\\n\u003ca class=\"\" href=\"/about/\" title=\"\"\u003eAbout\u003c/a\u003e\\n\u003cul aria-hidden=\"true\" class=\"subnav menu\" role=\"menu\"\u003e\\n\u003cli class=\"tier-2 element-1\" role=\"treeitem\"\u003e\u003ca href=\"/about/apps/\" title=\"\"\u003eApplications\u003c/a\u003e\u003c/li\u003e\\n\u003cli class=\"tier-2 element-2\" role=\"treeitem\"\u003e\u003ca href=\"/about/quotes/\" title=\"\"\u003eQuotes\u003c/a\u003e\u003c/li\u003e\\n\u003cli class=\"tier-2 element-3\" role=\"treeitem\"\u003e\u003ca href=\"/about/gettingstarted/\" title=\"\"\u003eGetting Started\u003c/a\u003e\u003c/li\u003e\\n\u003cli class=\"tier-2 element-4\" role=\"treeitem\"\u003e\u003ca href=\"/about/help/\" title=\"\"\u003eHelp\u003c/a\u003e\u003c/li\u003e\\n\u003cli class=\"tier-2 element-5\" role=\"treeitem\"\u003e\u003ca href=\"http://brochure.getpython.info/\" title=\"\"\u003ePython Brochure\u003c/a\u003e\u003c/li\u003e\\n\u003c/ul\u003e\\n\u003c/li\u003e'\n\n\n\nSelect Elements within Elements:\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e about.find('a')\n    [\u003cElement 'a' href='/about/' title='' class=''\u003e, \u003cElement 'a' href='/about/apps/' title=''\u003e, \u003cElement 'a' href='/about/quotes/' title=''\u003e, \u003cElement 'a' href='/about/gettingstarted/' title=''\u003e, \u003cElement 'a' href='/about/help/' title=''\u003e, \u003cElement 'a' href='http://brochure.getpython.info/' title=''\u003e]\n\nSearch for links within an element:\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e about.absolute_links\n    {'http://brochure.getpython.info/', 'https://www.python.org/about/gettingstarted/', 'https://www.python.org/about/', 'https://www.python.org/about/quotes/', 'https://www.python.org/about/help/', 'https://www.python.org/about/apps/'}\n\n\nSearch for text on the page:\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e r.html.search('Python is a {} language')[0]\n    programming\n\nMore complex CSS Selector example (copied from Chrome dev tools):\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e r = session.get('https://github.com/')\n    \u003e\u003e\u003e sel = 'body \u003e div.application-main \u003e div.jumbotron.jumbotron-codelines \u003e div \u003e div \u003e div.col-md-7.text-center.text-md-left \u003e p'\n    \u003e\u003e\u003e print(r.html.find(sel, first=True).text)\n    GitHub is a development platform inspired by the way you work. From open source to business, you can host and review code, manage projects, and build software alongside millions of other developers.\n\nXPath is also supported:\n\n.. code-block:: pycon\n\n   \u003e\u003e\u003e r.html.xpath('/html/body/div[1]/a')\n   [\u003cElement 'a' class=('px-2', 'py-4', 'show-on-focus', 'js-skip-to-content') href='#start-of-content' tabindex='1'\u003e]\n\n\nJavaScript Support\n==================\n\nLet's grab some text that's rendered by JavaScript. Until 2020, the Python 2.7 countdown clock (https://pythonclock.org) will serve as a good test page:\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e r = session.get('https://pythonclock.org')\n\nLet's try and see the dynamically rendered code (The countdown clock). To do that quickly at first, we'll search between the last text we see before it ('Python 2.7 will retire in...') and the first text we see after it ('Enable Guido Mode').\n\n.. code-block:: pycon\n\n\t\u003e\u003e\u003e r.html.search('Python 2.7 will retire in...{}Enable Guido Mode')[0]\n\t'\u003c/h1\u003e\\n        \u003c/div\u003e\\n        \u003cdiv class=\"python-27-clock\"\u003e\u003c/div\u003e\\n        \u003cdiv class=\"center\"\u003e\\n            \u003cdiv class=\"guido-button-block\"\u003e\\n                \u003cbutton class=\"js-guido-mode guido-button\"\u003e'\n\nNotice the clock is missing. The ``render()`` method takes the response and renders the dynamic content just like a web browser would.\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e r.html.render()\n    \u003e\u003e\u003e r.html.search('Python 2.7 will retire in...{}Enable Guido Mode')[0]\n    '\u003c/h1\u003e\\n        \u003c/div\u003e\\n        \u003cdiv class=\"python-27-clock is-countdown\"\u003e\u003cspan class=\"countdown-row countdown-show6\"\u003e\u003cspan class=\"countdown-section\"\u003e\u003cspan class=\"countdown-amount\"\u003e1\u003c/span\u003e\u003cspan class=\"countdown-period\"\u003eYear\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"countdown-section\"\u003e\u003cspan class=\"countdown-amount\"\u003e2\u003c/span\u003e\u003cspan class=\"countdown-period\"\u003eMonths\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"countdown-section\"\u003e\u003cspan class=\"countdown-amount\"\u003e28\u003c/span\u003e\u003cspan class=\"countdown-period\"\u003eDays\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"countdown-section\"\u003e\u003cspan class=\"countdown-amount\"\u003e16\u003c/span\u003e\u003cspan class=\"countdown-period\"\u003eHours\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"countdown-section\"\u003e\u003cspan class=\"countdown-amount\"\u003e52\u003c/span\u003e\u003cspan class=\"countdown-period\"\u003eMinutes\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"countdown-section\"\u003e\u003cspan class=\"countdown-amount\"\u003e46\u003c/span\u003e\u003cspan class=\"countdown-period\"\u003eSeconds\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/div\u003e\\n        \u003cdiv class=\"center\"\u003e\\n            \u003cdiv class=\"guido-button-block\"\u003e\\n                \u003cbutton class=\"js-guido-mode guido-button\"\u003e'\n\nLet's clean it up a bit. This step is not needed, it just makes it a bit easier to visualize the returned html to see what we need to target to extract our required information. \n\n.. code-block:: pycon\n\n\t\u003e\u003e\u003e from pprint import pprint\n\t\u003e\u003e\u003e pprint(r.html.search('Python 2.7 will retire in...{}Enable')[0])\n\t('\u003c/h1\u003e\\n'\n '        \u003c/div\u003e\\n'\n '        \u003cdiv class=\"python-27-clock is-countdown\"\u003e\u003cspan class=\"countdown-row '\n 'countdown-show6\"\u003e\u003cspan class=\"countdown-section\"\u003e\u003cspan '\n 'class=\"countdown-amount\"\u003e1\u003c/span\u003e\u003cspan '\n 'class=\"countdown-period\"\u003eYear\u003c/span\u003e\u003c/span\u003e\u003cspan '\n 'class=\"countdown-section\"\u003e\u003cspan class=\"countdown-amount\"\u003e2\u003c/span\u003e\u003cspan '\n 'class=\"countdown-period\"\u003eMonths\u003c/span\u003e\u003c/span\u003e\u003cspan '\n 'class=\"countdown-section\"\u003e\u003cspan class=\"countdown-amount\"\u003e28\u003c/span\u003e\u003cspan '\n 'class=\"countdown-period\"\u003eDays\u003c/span\u003e\u003c/span\u003e\u003cspan '\n 'class=\"countdown-section\"\u003e\u003cspan class=\"countdown-amount\"\u003e16\u003c/span\u003e\u003cspan '\n 'class=\"countdown-period\"\u003eHours\u003c/span\u003e\u003c/span\u003e\u003cspan '\n 'class=\"countdown-section\"\u003e\u003cspan class=\"countdown-amount\"\u003e52\u003c/span\u003e\u003cspan '\n 'class=\"countdown-period\"\u003eMinutes\u003c/span\u003e\u003c/span\u003e\u003cspan '\n 'class=\"countdown-section\"\u003e\u003cspan class=\"countdown-amount\"\u003e46\u003c/span\u003e\u003cspan '\n 'class=\"countdown-period\"\u003eSeconds\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/div\u003e\\n'\n '        \u003cdiv class=\"center\"\u003e\\n'\n '            \u003cdiv class=\"guido-button-block\"\u003e\\n'\n '                \u003cbutton class=\"js-guido-mode guido-button\"\u003e')\n\nThe rendered html has all the same methods and attributes as above. Let's extract just the data that we want out of the clock into something easy to use elsewhere and introspect like a dictionary.\n\n.. code-block:: pycon\n\t\n\t\u003e\u003e\u003e periods = [element.text for element in r.html.find('.countdown-period')]\n\t\u003e\u003e\u003e amounts = [element.text for element in r.html.find('.countdown-amount')]\n\t\u003e\u003e\u003e countdown_data = dict(zip(periods, amounts))\n\t\u003e\u003e\u003e countdown_data\n\t{'Year': '1', 'Months': '2', 'Days': '5', 'Hours': '23', 'Minutes': '34', 'Seconds': '37'}\n\nOr you can do this async also:\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e async def get_pyclock():\n    ...     r = await asession.get('https://pythonclock.org/')\n    ...     await r.html.arender()\n    ...     return r\n    ...\n    \u003e\u003e\u003e results = asession.run(get_pyclock, get_pyclock, get_pyclock)\n\nThe rest of the code operates the same way as the synchronous version except that ``results`` is a list containing multiple response objects however the same basic processes can be applied as above to extract the data you want. \n\nNote, the first time you ever run the ``render()`` method, it will download\nChromium into your home directory (e.g. ``~/.pyppeteer/``). This only happens\nonce.\n\nUsing without Requests\n======================\n\nYou can also use this library without Requests:\n\n.. code-block:: pycon\n\n    \u003e\u003e\u003e from requests_html import HTML\n    \u003e\u003e\u003e doc = \"\"\"\u003ca href='https://httpbin.org'\u003e\"\"\"\n    \u003e\u003e\u003e html = HTML(html=doc)\n    \u003e\u003e\u003e html.links\n    {'https://httpbin.org'}\n\n\nInstallation\n============\n\n.. code-block:: shell\n\n    $ pipenv install requests-html\n    ✨🍰✨\n\nOnly **Python 3.6 and above** is supported.\n","funding_links":[],"categories":["Python","HarmonyOS","Data Processing","Web内容提取","Web Content Extracting","网络服务","网络","scraping","Web Content Extracting [🔝](#readme)"],"sub_categories":["Windows Manager","Data Pre-processing \u0026 Loading","网络服务_其他"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpsf%2Frequests-html","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpsf%2Frequests-html","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpsf%2Frequests-html/lists"}