{"id":21071341,"url":"https://github.com/smartkeyerror/python-code-snippets","last_synced_at":"2025-09-07T06:06:01.208Z","repository":{"id":109331226,"uuid":"281585516","full_name":"SmartKeyerror/Python-Code-Snippets","owner":"SmartKeyerror","description":"Record special Python codes","archived":false,"fork":false,"pushed_at":"2020-07-23T01:35:39.000Z","size":5,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-03-14T02:47:45.692Z","etag":null,"topics":["python3"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SmartKeyerror.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-22T05:40:35.000Z","updated_at":"2021-06-21T16:03:02.000Z","dependencies_parsed_at":null,"dependency_job_id":"998cd9aa-a888-46f7-ac78-426460f798ed","html_url":"https://github.com/SmartKeyerror/Python-Code-Snippets","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/SmartKeyerror/Python-Code-Snippets","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SmartKeyerror%2FPython-Code-Snippets","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SmartKeyerror%2FPython-Code-Snippets/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SmartKeyerror%2FPython-Code-Snippets/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SmartKeyerror%2FPython-Code-Snippets/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SmartKeyerror","download_url":"https://codeload.github.com/SmartKeyerror/Python-Code-Snippets/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SmartKeyerror%2FPython-Code-Snippets/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274001418,"owners_count":25205254,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-07T02:00:09.463Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python3"],"created_at":"2024-11-19T18:52:03.443Z","updated_at":"2025-09-07T06:06:01.182Z","avatar_url":"https://github.com/SmartKeyerror.png","language":null,"readme":"# Python-Code-Snippets\nRecord special Python codes\n\n\n### 1. Use `singledispatch` transforms a function into a generic function\n\nThe `singledispatch` is a function decorator, which can transform a function into a generic function, that means we can use this to implement function overloading in python, like Java.\n\n```python\nfrom functools import singledispatch\n\n\n@singledispatch\ndef process(*args, **kwargs):\n    pass\n\n\n@process.register(str)\ndef broker(params):\n    print(\"str params: {}\".format(type(params)))\n\n\n@process.register(list)\ndef broker(params):\n    print(\"list params: {}\".format(type(params)))\n\n\n@process.register(dict)\ndef broker(params):\n    print(\"dict params: {}\".format(type(params)))\n\n\nif __name__ == \"__main__\":\n    process(\"foo\")\n    process([1, 2, 3])\n    process({\"foo\": \"bar\"})\n```\n\nWe can use this to make a function process any type of parameters and without `if` statement. It's mostly likes a \"Strategy Mode\".\n\n### 2. Use `lru_cache` to cache function result\n\nAs it's name means, `lru_cache` can save the results of time-consuming functions to avoid repeated calculations when the same parameters are passed in, like CPU cache，but in RAM.\n\n```python\nfrom functools import lru_cache\n\n@lru_cache(maxsize=256)\ndef factorial(n):\n    return n * factorial(n-1) if n != 1 else 1\n```\n\n### 3. Use Linux Signal to set a timeout for function execution\n\nIn UNIX API，`timer_create` can create a timer, which will send a SIGALRAM signal when expired, we can use this timer and `sigsetjmp`，`siglongjmp` to implements timeout function.\n\nPython module has UNIX Singal implements, **but it's just only works in main thread.**\n\n```python\nimport time\nimport signal\n\ndef handler(signum, frame):\n    raise TimeoutError\n\ndef with_timeout(timeout):\n    \"\"\"\n    This decorator is only works in main thread!\n    \"\"\"\n    def wrapper(func):\n        def set_timer(*args, **kwargs):\n            signal.signal(signal.SIGALRM, handler)  # Register a function to SIGALRM\n            signal.alarm(timeout)  # Set timer's expire time\n            result = func(*args, **kwargs)\n            signal.alarm(0)  # shutdown timer\n            return result\n        return set_timer\n    return wrapper\n\n@with_timeout(2)\ndef foo():\n    time.sleep(3)\n    print(\"biu~\")\n\nif __name__ == \"__main__\":\n    foo()\n```\n\n### 4. Use Linux Thread to set a timeout for function execution\n\nThere has a system-call which name is `pthread_join()` in POSIX, it wait for the end of another thread, here is the function prototype:\n\n```cpp\npthread_join(pthread_t thread, void **retval)\n```\n\npointer `**retval` is the result of another thread, in the other world, `pthread_join` can get another thread's execution result. But unfortunately, this system-call doesn't have timeout paramters.\n\nIn Python threading, we can use `Thread().join(timeout)` to wait for the end of another thread, but we can't get it's result, and we can specify `timeout` paramters, so we can use this feature to build our timeout-function.\n\n```python\nimport time\nfrom threading import Thread\n\n\nclass MaxTimeExceeded(Exception):\n    pass\n\n\ndef run_func(func, *args, **kwargs):\n    try:\n        return func(*args, **kwargs), None\n    except Exception as e:\n        return None, str(e)\n\n\ndef with_timeout(timeout):\n    def func_wrapper(func):\n        def wrapper(*args, **kwargs):\n\n            class SavedResultThread(Thread):\n                def __init__(self):\n                    super(SavedResultThread, self).__init__()\n                    self.result = None\n\n                def run(self) -\u003e None:\n                    self.result = run_func(func, *args, **kwargs)\n\n            t = SavedResultThread()\n            t.start()\n\n            t.join(timeout)\n\n            if t.is_alive():\n                # One thing, we can't stop this thread, it may be run forever\n                raise MaxTimeExceeded()\n\n            return t.result\n        return wrapper\n    return func_wrapper\n\n\nif __name__ == \"__main__\":\n\n    @with_timeout(2)\n    def test_func():\n        time.sleep(1)\n        return \"ok\"\n\n    result = test_func()\n    print(result)\n```\n\n### 5. Use bit shift to replace division\n\nIn binary search algorithm, we may use `start + (last - start) // 2` to calculate mid-index, but we can use bit shift to optimiz this calculation.\n\n```bash\n# python3\n\nIn [1]: 10 \u003e\u003e 1\nOut[1]: 5\n\nIn [2]: 9 \u003e\u003e 1\nOut[2]: 4\n```\n\nThe entire binary search implemention:\n\n```python\ndef binary_search(value, sorted_list):\n    start, last = 0, len(sorted_list) - 1\n    while start \u003c= last:\n        mid = start + ((last - start) \u003e\u003e 1)\n        if sorted_list[mid] == value: \n            return mid\n        if sorted_list[mid] \u003c value:\n            start = mid + 1\n        else:\n            last = mid - 1\n    return None\n```\n\nBy the way, also can use bit shift to calculate multiplication, such as $ m * 2^n $\n\n```bash\nIn [6]: 1 \u003c\u003c 1   # 1 * 2\nOut[6]: 2\n\nIn [7]: 5 \u003c\u003c 2   # 5 * 2^2\nOut[7]: 20\n```\n\n### 6. Create metaclass template\n\nIn some special area, such as ORM, Form-Validate, we need to create a metaclass to create our own class dynamically, so we can write a \"metaclass-template\" to reduce iterant labor.\n\n```python\nimport inspect\n\ndef is_instance_or_subclass(val, class_):\n    try:\n        return issubclass(val, class_)\n    except TypeError:\n        return isinstance(val, class_)\n        \n\ndef _get_fields(attrs, field_class):\n    fields = [\n        (field_name, attrs.get(field_name))\n        for field_name, field_value in list(attrs.items())\n        if is_instance_or_subclass(field_value, field_class)\n    ]\n    return fields\n    \n\ndef _get_fields_by_mro(klass, field_class):\n    # use __mro__ get all parents\n    mro = inspect.getmro(klass)\n    return sum(\n        (\n            _get_fields(\n                getattr(base, '_declared_fields', base.__dict__),\n                field_class,\n            )\n            for base in mro[:0:-1]\n        ),\n        [],\n    )\n    \n\nclass MetaclassTemplate(type):\n    def __new__(mcs, name, parents, attrs):\n        \n        # ignore MetaclassTemplate's child class\n        bases = [b for b in parents if isinstance(b, MetaclassTemplate)]\n        if not bases:\n            return super().__new__(mcs, name, parents, attrs)\n    \n        class_fields = _get_fields(attrs, SomeType)\n\n        klass = super(MetaclassTemplate, mcs).__new__(mcs, name, parents, attrs)\n\n        inherited_fields = _get_fields_by_mro(klass, SomeType)\n\n        # register _declared_fields to class object\n        klass._declared_fields = dict(class_fields + inherited_fields)\n\n        return klass\n```\n\n\n## 7. Use Exponential-Backoff with retry\n\n\u003e Exponential backoff is an algorithm that uses feedback to multiplicatively decrease the rate of some process, in order to gradually find an acceptable rate. \n\nWhen we use `requests` Python package to send a network request, or use gRPC to execute a remote procedure call, it may returns unexpected errors, GATEWAY TIMEOUT, ABORTED etc. if this network request is very significant, we need retry.\n\nHere is a gRPC retry with Exponential-Backoff algorithm:\n\n```python\nimport time\nfrom grpc import StatusCode, RpcError\n\n\n# define retry times with different situation\nMAX_RETRIES_BY_CODE = {\n    StatusCode.INTERNAL: 1,\n    StatusCode.ABORTED: 3,\n    StatusCode.UNAVAILABLE: 5,\n    StatusCode.DEADLINE_EXCEEDED: 5\n}\n\n# define MIN and MAX sleeping seconds\nMIN_SLEEPING = 0.015625\nMAX_SLEEPING = 1.0\n\n\nclass RetriesExceeded(Exception):\n    \"\"\"docstring for RetriesExceeded\"\"\"\n    pass\n\n\ndef retry(f):\n    def wraps(*args, **kwargs):\n        retries = 0\n        while True:\n            try:\n                return f(*args, **kwargs)\n            except RpcError as e:\n                # 使用e.code()获取响应码\n                code = e.code()\n                max_retries = MAX_RETRIES_BY_CODE.get(code)\n\n                if max_retries is None:\n                    raise e\n\n                if retries \u003e max_retries:\n                    raise RetriesExceeded(e)\n\n                back_off = min(MIN_SLEEPING * 2 ** retries, MAX_SLEEPING)\n\n                retries += 1\n                time.sleep(back_off)\n    return wraps\n```","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmartkeyerror%2Fpython-code-snippets","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsmartkeyerror%2Fpython-code-snippets","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmartkeyerror%2Fpython-code-snippets/lists"}