{"id":37072546,"url":"https://github.com/pykit3/pk3","last_synced_at":"2026-01-14T08:31:07.526Z","repository":{"id":128783574,"uuid":"244546566","full_name":"pykit3/pk3","owner":"pykit3","description":"super repo for pykit sub package","archived":false,"fork":false,"pushed_at":"2025-08-28T17:23:10.000Z","size":53,"stargazers_count":3,"open_issues_count":6,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-28T23:56:14.382Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pykit3.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-03-03T05:05:20.000Z","updated_at":"2025-08-28T17:23:13.000Z","dependencies_parsed_at":null,"dependency_job_id":"ab53a96f-4e02-4f48-9f85-16445656fd9e","html_url":"https://github.com/pykit3/pk3","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/pykit3/pk3","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pykit3%2Fpk3","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pykit3%2Fpk3/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pykit3%2Fpk3/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pykit3%2Fpk3/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pykit3","download_url":"https://codeload.github.com/pykit3/pk3/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pykit3%2Fpk3/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28414161,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:16:59.381Z","status":"ssl_error","status_checked_at":"2026-01-14T08:13:45.490Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-14T08:31:06.810Z","updated_at":"2026-01-14T08:31:07.506Z","avatar_url":"https://github.com/pykit3.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"**pykit3** is a collection of python3 modules that are loved by [s2][] developers.\n\n| Name | Desc |\n| :-- | :-- |\n| [k3awssign][] | A python lib is used for adding aws version 4 signature to request. |\n| [k3cacheable][] | Cache data which access frequently. |\n| [k3cat][] | Just like nix command cat or tail, it continuously scan a file line by line. |\n| [k3cgrouparch][] | This lib is used to set up cgroup directory tree according to configuration saved in zookeeper, and add pid to cgroup accordingly. |\n| [k3color][] | create colored text on terminal |\n| [k3confloader][] | k3confloader loads conf for other pykit3 modules |\n| [k3daemonize][] | It supplies a command line interface API to start/stop/restart a daemon. |\n| [k3dict][] | It provides with several dict operation functions. |\n| [k3down2][] | convert markdown segment into easy to transfer media sucha images. |\n| [k3etcd][] | A python client for Etcd https://github.com/coreos/etcd This module will only work correctly with the etcd server version 2.3.x or later. |\n| [k3fmt][] | It provides with several string operation functions. |\n| [k3fnmatch][] | Enhanced fnmatch with grouping regex and path transformation |\n| [k3fs][] | File-system Utilities |\n| [k3git][] | wrapper of git command-line |\n| [k3handy][] | handy alias of mostly used functions |\n| [k3heap][] | k3heap is a binary min heap implemented with reference |\n| [k3http][] | We find that 'httplib' must work in blocking mode and it can not have a timeout when recving response. |\n| [k3httpmultipart][] | This module provides some util methods to get multipart headers and body. |\n| [k3jobq][] | k3jobq processes a series of inputs with functions concurrently |\n| [k3log][] | k3log is a collection of log utilities. |\n| [k3logcollector][] | Scan log files on local machine, collector all interested logs, and send to somewhere for display. |\n| [k3math][] | k3math is a toy math impl |\n| [k3mime][] | This module provide some util methods to handle mime type. |\n| [k3modutil][] | Submodule Utilities. |\n| [k3net][] | Utility functions for network related operation. |\n| [k3num][] | Convert number to human readable format in a string. |\n| [k3pattern][] | Find common prefix of several `string`s, tuples of string, or other nested structure, recursively by default. |\n| [k3portlock][] | k3protlock is a cross-process lock that is implemented with `tcp` port binding. |\n| [k3priorityqueue][] | priorityQueue is a queue with priority support |\n| [k3proc][] | easy to use `Popen` |\n| [k3rangeset][] | segmented range which is represented in a list of sorted interleaving range. |\n| [k3redisutil][] | For using redis more easily. |\n| [k3shell][] | A python module to manage commands. |\n| [k3stopwatch][] | StopWatch - library for adding timers and tags in your code for performance monitoring |\n| [k3str][] | k3str is a collection of string utilities. |\n| [k3thread][] | utility to create thread. |\n| [k3time][] | Time convertion utils |\n| [k3txutil][] | A collection of helper functions to implement transactional operations. |\n| [k3ut][] | unittest util |\n| [k3utdocker][] | unit test for python-docker |\n| [k3utfjson][] | force `json.dump` and `json.load` in `utf-8` encoding. |\n| [k3wsjobd][] | This module is a gevent based websocket server. When the server receives a job description from a client, it runs that job asynchronously in a thread, and reports the progress back to the client periodically. |\n| [k3zkutil][] | Some helper function to make life easier with zookeeper. |\n\n\n`pk3` is the super repo for pykit3 sub module repos\nThis is just a container and does nothing.\n\n# Usage\n\n```\npip install xxx\n```\n\n\n# Development\n\n## Engineering practices\n\n- Start with the correctness requirements, ignore the performance impact until\n  the end. You'll usually write something faster by focusing on keeping things\n  minimal anyway.\n\n- Throw away what can't be done in a day of coding. when you rewrite it\n  tomorrow, it will be simpler.\n\n- Write code for human.\n\n    - Clarity comes first.\n    - Then the correctness.\n    - Then the performance(It is python, right?).\n\n    You are a smart guy. But your colleagues may not\n    be smart as you are.\n\n    **Do not write smart codes**.\n\n- Only write comment to explain **WHY**.\n  Improve the code to explain **HOW** and **WHAT**.\n\n## Developing workflow\n\nFollow the github flow:\n\n- Fork a repo\n- Hacking on it\n- Fire a pull request and then merge it to upstream.\n\n    - Rebase to upstream master before open a PR to reduce conflict.\n\n- Open an issue to describe what you gonna do before hacking on it:\n    Let everybody else know what you are doing.\n\n\n\n\n## Make changes to a module\n\n- `github-action` is used for CI, such as  https://github.com/pykit3/k3proc/actions .\n    The CI action is defined in `.github/workflows/python-package.yml`.\n    Most of the time you do not need to modify this file unless **you know what\n    you are doing**.\n\n- Testing is based on `pytest`: `pip install pytest` and `pytest`\n\n- Document is generated by `sphinx`.\n  Python docstring follows [google docstring style](https://www.sphinx-doc.org/en/1.5/ext/example_google.html).\n  Documenting building stuffs are all in `docs/` dir.\n\n  To build and test the doc: `make doc`.\n\n- Github meta data such as description and project labels are managed with\n  `.github/settings.yml`. see https://github.com/pykit3/gh-config for detail.\n\n- A module repo needs a `__name__ = \"k3proc\"` and `__version__ = \"0.1.2\"` in its\n  top level `__init__.py` so that building script generates the module name\n  automatically. A `__version__` must be in [semantic-version][] syntax.\n\n- Publish a module to `pip`:\n\n    - Update the version in `__init__.py`, e.g., to publish next version\n        `0.2.14` of `k3proc`, updata the `__version__` to `\"0.2.14\"`:\n\n        ```\n        # cat k3proc/__init__.py\n        ...\n        __version__ = \"0.2.13\"\n        __name__ = 'k3proc'\n        ...\n        ```\n    - Commit the version changes and `make release`.\n        This step creates a new tag of the current version.\n\n    - Pushing the created tag triggers an github action that\n        build a pip module and upload to `pypi.org`.\n\n    Then one can install the module(e.g. `k3proc`)\n    with `pip install k3proc` or with version specified:\n    `pip install k3proc==0.2.14`.\n\n\n## Module dir layout\n\nTake `k3proc` as example:\n\n```\n# github.com/pykit3/k3proc/\n\n.github/workflows/python-package.yml\n.github/workflows/python-publish.yml\n.github/dependabot.yml\n.github/settings.yml\n_building/\ndocs/\ntest/\n__init__.py\nLICENSE\nMakefile\nproc.py*\nREADME.md\nrequirements.txt\nsetup.py\n```\n\n- `.github/workflows/python-package.yml`: github-action for CI.\n\n- `.github/workflows/python-publish.yml`: github-action for publishing a pip package.\n\n- `.github/dependabot.yml`: dependency check config.\n\n- `.github/settings.yml`: defines project desc, labels etc. Just modify it and push.\n\n- `_building/`: building utils for build pip package and docs etc.\n\n- `docs/source/conf.py`: universal doc generating config. DO NOT MODIFY.\n\n- `docs/source/index.rst`: defines what module/class/function to generate doc.\n    UPDATE this file for every modification to the codes.\n\n- `test/`: unit test files.\n\n- `test/data`: data file for test, if required.\n\n- `__init__.py`: defines meta info about a module. such as name, version and\n    module doc. Export APIs to end user.\n\n- `LICENSE`\n\n- `proc.py`: your actual works are here. choose whatever a name as you wish.\n    It should be directly imported by `__init__.py` to expose APIs to end user.\n\n- `README.md`: readme in pykit3 is generated. DO NOT MODIFY IT. `make readme`\n    re-generates README.md by extracting docs and examples from codes.\n    See: `_building/build_readme.py`.\n\n- `requirements.txt`: the dependency pip. Some packages that are depended by\n    all modules are defined in `_building/requirements.txt`.\n\n- `setup.py`: script to publish the module. You do not need to modify this file.\n    It is auto generated with `make release`.\n\n\n## Create a new module\n\n- Fork from [tmpl][], which is a template module.\n    Choose a good name starts with `k3`, e.g. `k3whatever`.\n\n- Clone the repo to your laptop:\n  `git clone git@github.com:pykit3/k3whatever.git`.\n\n- Populate it: `python ./_building/populate.py`.\n    This step generates a skeleton of a module:\n    ```\n    python ./_building/populate.py\n    git status\n    ...\n        new file：   .github/settings.yml\n        new file：   __init__.py\n        new file：   docs/source/index.rst\n        new file：   package.json\n        new file：   packages.txt\n        new file：   requirements.txt\n        new file：   test-requirements.txt\n        new file：   test/test_doctest.py\n        new file：   test/test_k3whatever.py\n    ```\n\n    Examine the generated files and add them and make your first commit!\n\n## Document\n\nWrite the `f**king` doc!\n\n- Document the module, classes and methods.\n\n## Testing\n\nWrite the `f**king` test!\n\n\n[s2]: https://github.com/bsc-s2\n[semantic-version]: https://semver.org/\n[tmpl]: https://github.com/pykit3/tmpl\n\n[k3awssign]: https://github.com/pykit3/k3awssign\n[k3cacheable]: https://github.com/pykit3/k3cacheable\n[k3cat]: https://github.com/pykit3/k3cat\n[k3cgrouparch]: https://github.com/pykit3/k3cgrouparch\n[k3color]: https://github.com/pykit3/k3color\n[k3confloader]: https://github.com/pykit3/k3confloader\n[k3daemonize]: https://github.com/pykit3/k3daemonize\n[k3dict]: https://github.com/pykit3/k3dict\n[k3down2]: https://github.com/pykit3/k3down2\n[k3etcd]: https://github.com/pykit3/k3etcd\n[k3fmt]: https://github.com/pykit3/k3fmt\n[k3fnmatch]: https://github.com/pykit3/k3fnmatch\n[k3fs]: https://github.com/pykit3/k3fs\n[k3git]: https://github.com/pykit3/k3git\n[k3handy]: https://github.com/pykit3/k3handy\n[k3heap]: https://github.com/pykit3/k3heap\n[k3http]: https://github.com/pykit3/k3http\n[k3httpmultipart]: https://github.com/pykit3/k3httpmultipart\n[k3jobq]: https://github.com/pykit3/k3jobq\n[k3log]: https://github.com/pykit3/k3log\n[k3logcollector]: https://github.com/pykit3/k3logcollector\n[k3math]: https://github.com/pykit3/k3math\n[k3mime]: https://github.com/pykit3/k3mime\n[k3modutil]: https://github.com/pykit3/k3modutil\n[k3net]: https://github.com/pykit3/k3net\n[k3num]: https://github.com/pykit3/k3num\n[k3pattern]: https://github.com/pykit3/k3pattern\n[k3portlock]: https://github.com/pykit3/k3portlock\n[k3priorityqueue]: https://github.com/pykit3/k3priorityqueue\n[k3proc]: https://github.com/pykit3/k3proc\n[k3rangeset]: https://github.com/pykit3/k3rangeset\n[k3redisutil]: https://github.com/pykit3/k3redisutil\n[k3shell]: https://github.com/pykit3/k3shell\n[k3stopwatch]: https://github.com/pykit3/k3stopwatch\n[k3str]: https://github.com/pykit3/k3str\n[k3thread]: https://github.com/pykit3/k3thread\n[k3time]: https://github.com/pykit3/k3time\n[k3txutil]: https://github.com/pykit3/k3txutil\n[k3ut]: https://github.com/pykit3/k3ut\n[k3utdocker]: https://github.com/pykit3/k3utdocker\n[k3utfjson]: https://github.com/pykit3/k3utfjson\n[k3wsjobd]: https://github.com/pykit3/k3wsjobd\n[k3zkutil]: https://github.com/pykit3/k3zkutil","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpykit3%2Fpk3","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpykit3%2Fpk3","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpykit3%2Fpk3/lists"}