{"id":17030004,"url":"https://github.com/rob-blackbourn/asyncio-upgradeable-streams","last_synced_at":"2026-02-26T06:02:24.847Z","repository":{"id":57438129,"uuid":"430391026","full_name":"rob-blackbourn/asyncio-upgradeable-streams","owner":"rob-blackbourn","description":"An experiment in upgradeable streams in asyncio python","archived":false,"fork":false,"pushed_at":"2023-05-25T05:50:17.000Z","size":33,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-30T13:37:47.608Z","etag":null,"topics":["asyncio","python","ssl","starttls","tls"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rob-blackbourn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2021-11-21T14:34:34.000Z","updated_at":"2022-01-21T10:50:57.000Z","dependencies_parsed_at":"2022-09-06T08:10:21.735Z","dependency_job_id":null,"html_url":"https://github.com/rob-blackbourn/asyncio-upgradeable-streams","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/rob-blackbourn/asyncio-upgradeable-streams","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rob-blackbourn%2Fasyncio-upgradeable-streams","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rob-blackbourn%2Fasyncio-upgradeable-streams/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rob-blackbourn%2Fasyncio-upgradeable-streams/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rob-blackbourn%2Fasyncio-upgradeable-streams/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rob-blackbourn","download_url":"https://codeload.github.com/rob-blackbourn/asyncio-upgradeable-streams/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rob-blackbourn%2Fasyncio-upgradeable-streams/sbom","scorecard":{"id":779628,"data":{"date":"2025-08-11","repo":{"name":"github.com/rob-blackbourn/asyncio-upgradeable-streams","commit":"2d6494a3c710768411acf6f42482d275fe17e422"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.6,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Code-Review","score":0,"reason":"Found 0/26 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":0,"reason":"license file not detected","details":["Warn: project does not have a license file"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}}]},"last_synced_at":"2025-08-23T04:36:25.899Z","repository_id":57438129,"created_at":"2025-08-23T04:36:25.900Z","updated_at":"2025-08-23T04:36:25.900Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29849829,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-25T22:37:40.667Z","status":"online","status_checked_at":"2026-02-26T02:00:06.774Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","python","ssl","starttls","tls"],"created_at":"2024-10-14T08:03:23.655Z","updated_at":"2026-02-26T06:02:24.806Z","avatar_url":"https://github.com/rob-blackbourn.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# asyncio-upgradeable-streams\n\n## Update\n\nAs of Python 3.11 the\n[`start_tls`](https://docs.python.org/3/library/asyncio-stream.html#asyncio.StreamWriter.start_tls)\nfunctionality has been fixed.\n\nYou can find an example client and server (`client_3_11.py`, `server_3_11.py`)\nin the demos folder.\n\n\n---\n\n# Original README\n\nAn experiment in upgradeable streams.\n\n## Overview\n\nAn upgradeable stream starts life as a plain socket connection, but is capable\nof being \"upgraded\" to TLS. This is sometimes known as\n[STARTTLS](https://en.wikipedia.org/wiki/Opportunistic_TLS).\nCommon examples of this are\n[SMTP](https://datatracker.ietf.org/doc/html/rfc3207),\n[LDAP](https://datatracker.ietf.org/doc/html/rfc2830), and HTTP proxy tunneling\nwith [CONNECT](https://www.ietf.org/rfc/rfc2817.txt).\n\nThe asyncio library provides\n[loop.start_tls](https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.start_tls)\nfor this purpose, however there is little information on how this can be used.\n\nThis project provides an implementation of\n[asyncio.open_connection](https://docs.python.org/3/library/asyncio-stream.html#asyncio.open_connection)\nand [asyncio.start_server](https://docs.python.org/3/library/asyncio-stream.html#asyncio.start_server)\nwith an extra optional boolean parameter `upgradeadble`. When this is set the\nTLS negotiation is deferred, and the `writer` has a new method `start_tls` which\ncan be called to upgrade the connection to TLS.\n\nThis was tested using Python 3.9.7 on Ubuntu Linux 21.10.\n\n## Issues\n\nThe solution makes use of private variables in the python standard library which\nmay change at the whim of the python library maintainer. In particular it has\nto reset the reader in the `StreamReaderProtocol` and the transport in the\n`StreamWriter`.\n\n## Installation\n\nThis can be installed with pip.\n\n```bash\npip install jetblack-upgradeable-streams\n```\n\n## Examples\n\nThe following examples can be found in the \"demos\" folder. They expect a Linux\nenvironment.\n\n### Client\n\nA new argument `upgradeable` has been added to the\n`open_connection` function to enable upgrading. When `upgradeable` is `True`\nthe TLS negotiation is deferred and the `ssl` parameter is stored for use when\nthe connection is upgraded.\nThe `writer` has a new method `start_tls` to upgrade the connection to TLS.\n\n1. The client connects with the `upgradeable` argument set to `True` and an\n   `ssl` context. The TLS negotiation will be deferred until `start_tls` is\n   called on the `writer`.\n\n2. First the client sends \"PING\" to the server. The server should respond\n   with \"PONG\".\n\n3. Next the client sends \"STARTTLS\" to instruct the server to upgrade the\n   connection to TLS. The client then calls the `start_tls` method on the\n   `writer` to negotiate the secure connection. The method returns a new\n   `reader` and `writer`.\n\n4. Using the new writer the client sends \"PING\" to the server, this time over\n   the encrypted stream. The server should respond with \"PONG\".\n\n5. Finally the client sends \"QUIT\" to the server and closes the connection.\n\n```python\nimport asyncio\nimport socket\nimport ssl\n\nfrom upgradeable_streams import open_connection\n\n\nasync def start_client():\n    ctx = ssl.create_default_context(\n        purpose=ssl.Purpose.SERVER_AUTH,\n        cafile='/etc/ssl/certs/ca-certificates.crt'\n    )\n    host = socket.getfqdn()\n\n    print(\"Connect to server as upgradeable\")\n    reader, writer = await open_connection(\n        host,\n        10001,\n        ssl=ctx,\n        upgradeable=True\n    )\n\n    print(f\"The writer ssl context is {writer.get_extra_info('sslcontext')}\")\n\n    print(\"Sending PING\")\n    writer.write(b'PING\\n')\n    response = (await reader.readline()).decode('utf-8').rstrip()\n    print(f\"Received: {response}\")\n\n    print(\"Sending STARTTLS\")\n    writer.write(b'STARTTLS\\n')\n\n    print(\"Upgrading the connection\")\n    # Upgrade\n    reader, writer = await writer.start_tls()\n\n    print(f\"The writer ssl context is {writer.get_extra_info('sslcontext')}\")\n\n    print(\"Sending PING\")\n    writer.write(b'PING\\n')\n    response = (await reader.readline()).decode('utf-8').rstrip()\n    print(f\"Received: {response}\")\n\n    print(\"Sending QUIT\")\n    writer.write(b'QUIT\\n')\n    await writer.drain()\n\n    print(\"Closing client\")\n    writer.close()\n    await writer.wait_closed()\n    print(\"Client disconnected\")\n\nif __name__ == '__main__':\n    asyncio.run(start_client())\n```\n\n### Server\n\nAn extra argument `upgradeable` has been added to the `start_server` function\nto enable upgrading to TLS. The `ssl` context is stored for use when a client\nconnection is upgraded to TLS.\nThe `writer` has a new method `start_tls` to upgrade the connection to TLS.\n\n1. The server starts and listens for client connections. The `upgradeable` flag\n   is set to `True` and the `ssl` context is provided. The client connections\n   will start without TLS, but can be upgraded by calling `start_tls`.\n\n2. On receiving a connection the client callback enters a read loop.\n\n3. When the server receives \"PING\" it responds with \"PONG\".\n\n4. When the server receives \"STARTTLS\" it calls the `start_tls` method on the\n   `writer` to negotiate the TLS connection. The method returns a new `reader`\n   and `writer`.\n\n5. When the server receives \"QUIT\" it closes the connection.\n\nThe code expects certificate and key PEM files in \"~/.keys/server.{crt,key}\".\n\n```python\nimport asyncio\nfrom asyncio import StreamReader, StreamWriter\nfrom os.path import expanduser\nimport socket\nimport ssl\nfrom typing import Union\n\nfrom upgradeable_streams import start_server, UpgradeableStreamWriter\n\n\nasync def handle_client(\n        reader: StreamReader,\n        writer: Union[UpgradeableStreamWriter, StreamWriter]\n) -\u003e None:\n    print(\"Client connected\")\n\n    while True:\n        request = (await reader.readline()).decode('utf8').rstrip()\n        print(f\"Read '{request}'\")\n\n        if request == 'QUIT':\n            break\n\n        elif request == 'PING':\n            print(\"Sending pong\")\n            writer.write(b'PONG\\n')\n            await writer.drain()\n\n        elif request == 'STARTTLS':\n            if not isinstance(writer, UpgradeableStreamWriter):\n                raise ValueError('writer not upgradeable')\n            print(\"Upgrading connection to TLS\")\n            # Upgrade\n            reader, writer = await writer.start_tls()\n\n    print(\"Closing client\")\n    writer.close()\n    await writer.wait_closed()\n    print(\"Client closed\")\n\n\nasync def run_server():\n    ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)\n    ctx.load_verify_locations(cafile=\"/etc/ssl/certs/ca-certificates.crt\")\n    ctx.load_cert_chain(\n        expanduser(\"~/.keys/server.crt\"),\n        expanduser(\"~/.keys/server.key\")\n    )\n    host = socket.getfqdn()\n\n    print(\"Starting server as upgradeable\")\n    server = await start_server(\n        handle_client,\n        host,\n        10001,\n        ssl=ctx,\n        upgradeable=True\n    )\n\n    async with server:\n        await server.serve_forever()\n\nif __name__ == '__main__':\n    asyncio.run(run_server())\n```\n\n## Development\n\nPull requests are welcome. In particular anything to reduce the reliance on the\nimplementation details in the standard library.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frob-blackbourn%2Fasyncio-upgradeable-streams","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frob-blackbourn%2Fasyncio-upgradeable-streams","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frob-blackbourn%2Fasyncio-upgradeable-streams/lists"}