{"id":18296977,"url":"https://github.com/davidcellis/ducktools-lazyimporter","last_synced_at":"2025-06-18T10:38:46.332Z","repository":{"id":202634042,"uuid":"700265476","full_name":"DavidCEllis/ducktools-lazyimporter","owner":"DavidCEllis","description":"Lazy Import class for Python","archived":false,"fork":false,"pushed_at":"2025-06-16T17:15:26.000Z","size":230,"stargazers_count":1,"open_issues_count":3,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-16T17:55:23.716Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://ducktools-lazyimporter.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DavidCEllis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-10-04T09:11:12.000Z","updated_at":"2025-05-01T14:19:41.000Z","dependencies_parsed_at":"2023-11-12T22:27:14.545Z","dependency_job_id":"d75cb2d1-89d3-46e0-8dda-0df2103a2bb0","html_url":"https://github.com/DavidCEllis/ducktools-lazyimporter","commit_stats":null,"previous_names":["davidcellis/ducktools-lazyimporter"],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/DavidCEllis/ducktools-lazyimporter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidCEllis%2Fducktools-lazyimporter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidCEllis%2Fducktools-lazyimporter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidCEllis%2Fducktools-lazyimporter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidCEllis%2Fducktools-lazyimporter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DavidCEllis","download_url":"https://codeload.github.com/DavidCEllis/ducktools-lazyimporter/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidCEllis%2Fducktools-lazyimporter/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260536768,"owners_count":23024456,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T14:45:06.942Z","updated_at":"2025-06-18T10:38:46.324Z","avatar_url":"https://github.com/DavidCEllis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ducktools: lazyimporter #\n\nCreate an object to handle lazily importing from other modules.\n\nNearly every form of \"lazyimporter\" module name is taken on PyPI so this is namespaced.\n\nIntended to help save on start time where some modules are only needed for specific\nfunctions while allowing information showing the import information to appear at\nthe top of a module where expected.\n\nThis form of import works by creating a specific LazyImporter object that lazily\nimports modules or module attributes when the module or attribute is accessed\non the object.\n\n## How to download ##\n\nDownload from PyPI:\n    `python -m pip install ducktools-lazyimporter`\n\n## Example ##\n\nExample using the packaging module.\n\n```python\n__version__ = \"v0.1.5\"\n\nfrom ducktools.lazyimporter import LazyImporter, FromImport\n\nlaz = LazyImporter([\n    FromImport(\"packaging.version\", \"Version\")\n])\n\ndef is_newer_version(version_no: str) -\u003e bool:\n    \"\"\"Check if a version number given indicates \n    a newer version than this package.\"\"\"\n    this_ver = laz.Version(__version__) \n    new_ver = laz.Version(version_no)\n    return new_ver \u003e this_ver\n\n# Import will only occur when the function is called and \n# laz.Version is accessed\nprint(is_newer_version(\"v0.2.0\"))\n```\n\n## Why use a lazy importer? ##\n\nOne obvious use case is if you are creating a simple CLI application that you wish to feel fast.\nIf the application has multiple pathways a lazy importer can improve performance by avoiding\nloading the modules that are only needed for heavier pathways. (It may also be worth looking\nat what library you are using for CLI argument parsing.)\n\nI created this so I could use it on my own projects so here's an example of the performance\nof getting the help menu for `ducktools-env` with and without lazy imports.\n\nWith lazy imports:\n```commandline\nhyperfine -w3 -r20 \"python -m ducktools.env --help\"\n```\n```\nBenchmark 1: python -m ducktools.env --help\n  Time (mean ± σ):      41.4 ms ±   1.0 ms    [User: 21.1 ms, System: 15.9 ms]\n  Range (min … max):    40.0 ms …  44.1 ms    20 runs\n```\n\nWithout lazy imports (by setting `DUCKTOOLS_EAGER_IMPORT=true`):\n```commandline\nhyperfine -w3 -r20 \"python -m ducktools.env --help\"\n```\n```\nBenchmark 1: python -m ducktools.env --help\n  Time (mean ± σ):     112.8 ms ±   2.6 ms    [User: 78.1 ms, System: 35.9 ms]\n  Range (min … max):   109.2 ms … 117.8 ms    20 runs\n```\n\n## Hasn't this already been done ##\n\nYes.\n\nBut...\n\nMost implementations rely on stdlib modules that are themselves slow to import\n(for example: typing, importlib.util, logging, inspect, ast).\nBy contrast `ducktools-lazyimporter` only uses modules that python imports on launch.\n\n`ducktools-lazyimporter` does not attempt to propagate laziness, only the modules provided\nto `ducktools-lazyimporter` directly will be imported lazily. Any subdependencies of those \nmodules will be imported eagerly as if the import statement is placed where the \nimporter attribute is first accessed. \n\n## Use Case ##\n\nThere are two main use cases this is designed for.\n\n### Replacing in-line imports used in a module ###\n\nSometimes it is useful to use tools from a module that has a significant import time.\nIf this is part of a function/method that won't necessarily always be used it is \ncommon to delay the import and place it inside the function/method.\n\nRegular import within function:\n```python\ndef get_copy(obj):\n    from copy import deepcopy\n    return deepcopy(obj)\n```\n\nWith a LazyImporter:\n```python\nfrom ducktools.lazyimporter import LazyImporter, FromImport\n\nlaz = LazyImporter([FromImport(\"copy\", \"deepcopy\")])\n\ndef get_copy(obj):\n    return laz.deepcopy(obj)\n```\n\nWhile the LazyImporter is more verbose, it only invokes the import mechanism\nonce when first accessed, while placing the import within the function invokes\nit every time the function is called. This can be a significant overhead if\nthe function ends up used in a loop.\n\nThis also means that if the attribute is accessed anywhere it will be imported\nand in place wherever it is used.\n\n### Delaying the import of parts of a module's public API ###\n\nEager import:\n```python\nfrom .submodule import useful_tool\n\n__all__ = [..., \"useful_tool\"]\n```\n\nLazy import:\n```python\nfrom ducktools.lazyimporter import LazyImporter, FromImport, get_module_funcs\n\n__all__ = [..., \"useful_tool\"]\n\nlaz = LazyImporter(\n    [FromImport(\".submodule\", \"useful_tool\")],\n    globs=globals(),  # globals() is used for relative imports, LazyImporter will attempt to infer it if not provided\n)\n__getattr__, __dir__ = get_module_funcs(laz, __name__)  # __name__ will also be inferred if not given\n```\n\n## The import classes ##\n\nIn all of these instances `modules` is intended as the first argument\nto `LazyImporter` and all attributes would be accessed from the \n`LazyImporter` instance and not in the global namespace.\n\neg:\n```python\nfrom ducktools.lazyimporter import LazyImporter, ModuleImport\n\nmodules = [ModuleImport(\"functools\")]\nlaz = LazyImporter(modules)\nlaz.functools  # provides access to the module \"functools\"\n```\n\n### ModuleImport ###\n\n`ModuleImport` is used for your basic module style imports.\n\n```python\nfrom ducktools.lazyimporter import ModuleImport\n\nmodules = [\n    ModuleImport(\"module\"),\n    ModuleImport(\"other_module\", \"other_name\"),\n    ModuleImport(\"base_module.submodule\", asname=\"short_name\"),\n]\n```\n\nis equivalent to \n\n```python\nimport module\nimport other_module as other_name\nimport base_module.submodule as short_name\n```\n\nwhen provided to a LazyImporter and accessed as follows:\n\n```python\nfrom ducktools.lazyimporter import LazyImporter\nlaz = LazyImporter(modules)\n\nlaz.module  # module\nlaz.other_name  # other_module\nlaz.short_name  # base_module.submodule\n```\n\n### FromImport and MultiFromImport ###\n\n`FromImport` is used for standard 'from' imports, `MultiFromImport` for importing\nmultiple items from the same module. By using a `MultiFromImport`, when the first\nattribute is accessed, all will be assigned on the LazyImporter.\n\n```python\nfrom ducktools.lazyimporter import FromImport, MultiFromImport\n\nmodules = [\n    FromImport(\"dataclasses\", \"dataclass\"),\n    FromImport(\"functools\", \"partial\", \"partfunc\"),\n    MultiFromImport(\"collections\", [\"namedtuple\", (\"defaultdict\", \"dd\")]),\n]\n```\n\nis equivalent to\n\n```python\nfrom dataclasses import dataclass\nfrom functools import partial as partfunc\nfrom collections import namedtuple, defaultdict as dd\n```\n\nwhen provided to a LazyImporter and accessed as follows:\n\n```python\nfrom ducktools.lazyimporter import LazyImporter\nlaz = LazyImporter(modules)\n\nlaz.dataclass  # dataclasses.dataclass\nlaz.partfunc  # functools.partial\nlaz.namedtuple  # collections.namedtuple\nlaz.dd  # collections.defaultdict\n```\n\n### TryExceptImport, TryExceptFromImport and TryFallbackImport ###\n\n`TryExceptImport` is used for compatibility where a module may not be available\nand so a fallback module providing the same functionality should be used. For\nexample when a newer version of python has a stdlib module that has replaced\na third party module that was used previously.\n\n```python\nfrom ducktools.lazyimporter import TryExceptImport, TryExceptFromImport, TryFallbackImport\n\nmodules = [\n    TryExceptImport(\"tomllib\", \"tomli\", \"tomllib\"),\n    TryExceptFromImport(\"tomllib\", \"loads\", \"tomli\", \"loads\", \"loads\"),\n    TryFallbackImport(\"tomli\", None),\n]\n```\n\nis roughly equivalent to\n\n```python\ntry:\n    import tomllib as tomllib\nexcept ImportError:\n    import tomli as tomllib\n\ntry:\n    from tomllib import loads as loads\nexcept ImportError:\n    from tomli import loads as loads\n\ntry:\n    import tomli\nexcept ImportError:\n    tomli = None\n```\n\nwhen provided to a LazyImporter and accessed as follows:\n\n```python\nfrom ducktools.lazyimporter import LazyImporter\nlaz = LazyImporter(modules)\n\nlaz.tomllib  # tomllib / tomli\nlaz.loads  # tomllib.loads / tomli.loads\nlaz.tomli  # tomli / None\n```\n\n## Experimental import statement capture ##\n\nThere is an **experimental** mode that can capture import statements within a context block.\n\nThis is currently in a separate 'capture' submodule but may be merged (or lazily imported itself) in the future.\n\n```python\nfrom ducktools.lazyimporter import LazyImporter, get_importer_state\nfrom ducktools.lazyimporter.capture import capture_imports\n\nlaz = LazyImporter()\n\nwith capture_imports(laz, auto_export=True):\n    # Inside this block, imports are captured and converted to lazy imports on laz\n    import functools\n    from collections import namedtuple as nt\n\nprint(get_importer_state(laz))\n\n# Note that the captured imports are *not* available in the module namespace\ntry:\n    functools\nexcept NameError:\n    print(\"functools is not here\")\n```\n\nImports are placed on the lazy importer object as with the explicit syntax. Unlike the regular\nsyntax, these imports are exported by default.\n\nThis works by replacing and restoring the builtin `__import__` function that is called by the\nimport statement while in the block. \n\n### Context Manager Caveats ###\n\n* This only supports Module imports and From imports\n  * The actual statement executes immediately and returns a placeholder, so a try/except can't work.\n* Imports triggered inside functions or classes while within the block will still occur eagerly\n* Imports triggered in other modules while within the block will still occur eagerly\n* The context manager must be used at the module level\n  * It will error if you use it inside a class or function scope\n* As with the `ModuleImport` class, submodule imports without an assigned name are not supported.\n* If other modules are also replacing `__import__` **simultaneously** this will probably fail.\n  * In a library you may not be able to guarantee this.\n  * Hopefully this will be resolvable.\n\n## Environment Variables ##\n\nThere are two environment variables that can be used to modify the behaviour for\ndebugging purposes.\n\nIf `DUCKTOOLS_EAGER_PROCESS` is set to any value other than 'False' (case insensitive)\nthe initial processing of imports will be done on instance creation.\n\nSimilarly if `DUCKTOOLS_EAGER_IMPORT` is set to any value other than 'False' all imports\nwill be performed eagerly on instance creation (this will also force processing on import).\n\nIf they are unset this is equivalent to being set to False.\n\nIf there is a lazy importer where it is known this will not work \n(for instance if it is managing a circular dependency issue)\nthese can be overridden for an importer by passing values to `eager_process` and/or \n`eager_import` arguments to the `LazyImporter` constructer as keyword arguments.\n\n## How does it work ##\n\nThe following lazy importer:\n\n```python\nfrom ducktools.lazyimporter import LazyImporter, FromImport\n\nlaz = LazyImporter([FromImport(\"functools\", \"partial\")])\n```\n\nGenerates an object that's roughly equivalent to this:\n\n```python\nclass SpecificLazyImporter:\n    def __getattr__(self, name):\n        if name == \"partial\":\n            from functools import partial\n            setattr(self, name, partial)\n            return partial\n        \n        raise AttributeError(...)\n\nlaz = SpecificLazyImporter()\n```\n\nThe first time the attribute is accessed the import is done and the output\nis stored on the instance, so repeated access immediately gets the desired \nobject and the import mechanism is only invoked once.\n\n(The actual `__getattr__` function uses a dictionary lookup and delegates importing\nto the FromImport class. Names are all dynamic and imports are done through\nthe `__import__` function.)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidcellis%2Fducktools-lazyimporter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdavidcellis%2Fducktools-lazyimporter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidcellis%2Fducktools-lazyimporter/lists"}