{"id":17720359,"url":"https://github.com/gateway-experiments/hadoop-yarn-api-python-client","last_synced_at":"2026-03-09T08:02:03.564Z","repository":{"id":17244515,"uuid":"20013738","full_name":"gateway-experiments/hadoop-yarn-api-python-client","owner":"gateway-experiments","description":"Python client for Hadoop® YARN API","archived":false,"fork":false,"pushed_at":"2022-09-26T21:54:46.000Z","size":181,"stargazers_count":109,"open_issues_count":6,"forks_count":48,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-12-13T03:24:28.048Z","etag":null,"topics":["hacktoberfest","hadoop","yarn"],"latest_commit_sha":null,"homepage":"https://yarn-api-client-python.readthedocs.io/en/latest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gateway-experiments.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-05-21T08:25:35.000Z","updated_at":"2024-09-11T18:27:29.000Z","dependencies_parsed_at":"2022-09-06T08:10:43.732Z","dependency_job_id":null,"html_url":"https://github.com/gateway-experiments/hadoop-yarn-api-python-client","commit_stats":null,"previous_names":["codait/hadoop-yarn-api-python-client"],"tags_count":19,"template":false,"template_full_name":null,"purl":"pkg:github/gateway-experiments/hadoop-yarn-api-python-client","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gateway-experiments%2Fhadoop-yarn-api-python-client","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gateway-experiments%2Fhadoop-yarn-api-python-client/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gateway-experiments%2Fhadoop-yarn-api-python-client/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gateway-experiments%2Fhadoop-yarn-api-python-client/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gateway-experiments","download_url":"https://codeload.github.com/gateway-experiments/hadoop-yarn-api-python-client/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gateway-experiments%2Fhadoop-yarn-api-python-client/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30287446,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-09T02:57:19.223Z","status":"ssl_error","status_checked_at":"2026-03-09T02:56:26.373Z","response_time":61,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest","hadoop","yarn"],"created_at":"2024-10-25T15:26:43.571Z","updated_at":"2026-03-09T08:02:03.527Z","avatar_url":"https://github.com/gateway-experiments.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# hadoop-yarn-api-python-client\n\nPython client for Apache Hadoop® YARN API\n\n[![Latest Version](https://img.shields.io/pypi/v/yarn-api-client.svg)](https://pypi.python.org/pypi/yarn-api-client/)\n[![Downloads](https://pepy.tech/badge/yarn-api-client/month)](https://pepy.tech/project/yarn-api-client/month)\n[![Travis CI build status](https://travis-ci.org/CODAIT/hadoop-yarn-api-python-client.svg?branch=master)](https://travis-ci.org/CODAIT/hadoop-yarn-api-python-client)\n[![Latest documentation status](https://readthedocs.org/projects/yarn-api-client-python/badge/?version=latest)](https://yarn-api-client-python.readthedocs.org/en/latest/?badge=latest)\n[![Test coverage](https://coveralls.io/repos/toidi/hadoop-yarn-api-python-client/badge.png)](https://coveralls.io/r/toidi/hadoop-yarn-api-python-client)\n\nPackage documentation:\n[yarn-api-client-python.readthedocs.org](https://yarn-api-client-python.readthedocs.org/en/latest/)\n\nREST API documentation: [hadoop.apache.org](http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html)\n\n**Warning**: CLI is outdated \u0026 broken. Please don't use CLI. This will be resolved in future releases.\n\n---\n## Compatibility Matrix\n\n| yarn-api-client-python | Apache Hadoop |\n| ------------- | ------------- |\n| 1.0.2  | 3.2.1  |\n| 1.0.3  | 3.3.0, 3.3.1  |\n\nIf u have version other than mentioned (or vendored variant like Hortonworks), certain APIs might be not working or have differences in\nimplementation. If u plan to use certain API long-term, you might want to make sure its not in Alpha stage in documentation.\n\n## Installation\n\nFrom PyPI\n```\npip install yarn-api-client\n```\n\nFrom Anaconda (conda forge)\n```\nconda install -c conda-forge yarn-api-client\n```\n\nFrom source code\n```\npip install git+https://github.com/CODAIT/hadoop-yarn-api-python-client.git\n```\n\n## Enabling support for SimpleAuth\n\nSee example below:\n```\nfrom yarn_api_client.auth import SimpleAuth\nfrom yarn_api_client.history_server import HistoryServer\nauth = SimpleAuth('impersonated_account_name')\nhistory_server = HistoryServer('https://127.0.0.2:5678', auth=auth)\n```\n\n## Enabling support for Kerberos/SPNEGO Security\n1. First option - using `requests_kerberos` package  \n\nTo avoid deployment issues on a non Kerberized environment, the `requests_kerberos`\ndependency is optional and needs to be explicit installed in order to enable access\nto YARN console protected by Kerberos/SPNEGO.\n\n`pip install requests_kerberos`\n\nFrom python code\n```\nfrom yarn_api_client.history_server import HistoryServer\nfrom requests_kerberos import HTTPKerberosAuth\nhistory_server = HistoryServer('https://127.0.0.2:5678', auth=HTTPKerberosAuth())\n```\n\nPS: You **need** to get valid kerberos ticket in systemwide kerberos cache before running your code, otherwise calls to kerberized environment won't go through (run kinit before proceeding to run code)\n\n2. Second option - using `gssapi` package  \n\nIf you want to avoid using terminal calls, you have to perform SPNEGO handshake to retrieve ticket yourself. Full API documentation: https://pythongssapi.github.io/python-gssapi/latest/\n\n# Usage\n\n### CLI interface\n\n**Warning**: CLI is outdated \u0026 broken. Please don't use CLI. This will be resolved in future releases.\n\n1. First way\n```\nbin/yarn_client --help\n```\n\n2. Alternative way\n```\npython -m yarn_api_client --help\n```\n\n### Programmatic interface\n\n```\nfrom yarn_api_client import ApplicationMaster, HistoryServer, NodeManager, ResourceManager\nam = ApplicationMaster('https://127.0.0.2:5678')\napp_information = am.application_information('application_id')\n```\n\n### Changelog\n\n1.0.3 Release\n   - Drop support of Python 2.7 (if you still need it for extreme emergency, look into reverting ab4f71582f8c69e908db93905485ba4d00562dfd)\n   - Update of supported hadoop version to 3.3.1\n   - Add support for YARN_CONF_DIR and HADOOP_CONF_DIR\n   - Add class for native SimpleAuth (#106)\n   - Add constructor argument for proxies (#109)\n\n1.0.2 Release\n   - Add support for Python 3.8.x\n   - Fix HTTPS url parsing\n   - Fix JSON body request APIs\n   - Handle YARN response with empty contents\n   - Better logging support\n\n1.0.1 Release\n   - Passes the authorization instance to the Active RM check\n   - Establishes a new (working) documentation site in readthedocs.io: yarn-api-client-python.readthedocs.io\n   - Adds more python version (3.7 and 3.8) to test matrix and removes 2.6.\n\n1.0.0 Release\n   - Major cleanup of API.  \n     - Address/port parameters have been replaced with complete\n       endpoints (includes scheme [e.g., http or https]).\n     - ResourceManager has been updated to take a list of endpoints for\n       improved HA support.\n     - ResourceManager, ApplicationMaster, HistoryServer and NodeManager\n       have been updated with methods corresponding to the latest REST API.\n   - pytest support on Windows has been provided.\n   - Documentation has been updated.\n\n   **NOTE:** Applications using APIs relative to releases prior to 1.0 should\n   pin their dependency on yarn-api-client to _less than_ 1.0 and are encouraged\n   to update to 1.0 as soon as possible.\n\n0.3.7 Release  \n   - Honor configured HTTP Policy when no address is provided - enabling\n     using of HTTPS in these cases.\n\n0.3.6 Release  \n   - Extend ResourceManager to allow applications to determine\n     resource availability prior to submission.\n\n0.3.5 Release  \n   - Hotfix release to fix internal signature mismatch\n\n0.3.4 Release  \n   - More flexible support for discovering Hadoop configuration\n     including multiple Resource Managers when HA is configured\n   - Properly support YARN post response codes\n\n0.3.3 Release  \n   - Properly set Content-Type in PUT requests\n   - Check for HADOOP_CONF_DIR env variable\n\n0.3.2 Release  \n   - Make Kerberos/SPNEGO dependency optional\n\n0.3.1 Release  \n   - Fix cluster_application_kill API\n\n0.3.0 Release  \n   - Add support for YARN endpoints protected by Kerberos/SPNEGO\n   - Moved to `requests` package for REST API invocation\n   - Remove `http_con` property, as connections are now managed by `requests` package\n\n0.2.5 Release  \n  - Fixed History REST API\n\n0.2.4 Release  \n  - Added compatibility with HA enabled Resource Manager\n\n### Team\n\nYARN API client is developed by an open community, and the current maintainers \nare listed below in alphabetical order:\n\n- [Dmitry Romanenko](https://github.com/dimon222)\n- [Eduard Iskandarov](https://github.com/toidi)\n- [Kevin Bates](https://github.com/kevin-bates)\n- [Luciano Resende](https://github.com/lresende)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgateway-experiments%2Fhadoop-yarn-api-python-client","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgateway-experiments%2Fhadoop-yarn-api-python-client","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgateway-experiments%2Fhadoop-yarn-api-python-client/lists"}