{"id":16251511,"url":"https://github.com/pfalcon/python-imphook","last_synced_at":"2025-03-19T20:30:44.856Z","repository":{"id":62570748,"uuid":"323453811","full_name":"pfalcon/python-imphook","owner":"pfalcon","description":"Simple and clear import hooks for Python - import anything as if it were a Python module","archived":false,"fork":false,"pushed_at":"2021-12-10T20:40:45.000Z","size":51,"stargazers_count":38,"open_issues_count":0,"forks_count":3,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-02-28T20:46:34.742Z","etag":null,"topics":["import","import-hook","import-hooks","importlib","python"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/imphook/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pfalcon.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-21T21:39:44.000Z","updated_at":"2024-10-15T02:32:41.000Z","dependencies_parsed_at":"2022-11-03T18:25:58.193Z","dependency_job_id":null,"html_url":"https://github.com/pfalcon/python-imphook","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pfalcon%2Fpython-imphook","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pfalcon%2Fpython-imphook/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pfalcon%2Fpython-imphook/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pfalcon%2Fpython-imphook/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pfalcon","download_url":"https://codeload.github.com/pfalcon/python-imphook/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244014113,"owners_count":20383716,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["import","import-hook","import-hooks","importlib","python"],"created_at":"2024-10-10T15:10:26.615Z","updated_at":"2025-03-19T20:30:44.502Z","avatar_url":"https://github.com/pfalcon.png","language":"Python","readme":"imphook - Simple and clear import hooks for Python\n==================================================\n\nThe ``imphook`` module allows to easily define per file type import\nhooks, i.e. overload or extend import processing for particular file\ntypes, without affecting processing of other file types, and at the\nsame time, while ensuring that new processing integrates as seamlessly\nas possible with normal Python import rules.\n\nBesides the Python-level API to install import hooks, the module also\nprovides command-line interface to run an existing Python script or\nmodule with one or more import hooks preloaded (i.e. without modifying\nexisting script source code).\n\nSome but not all things you can easily do using ``imphook`` (most\nof these require additional modules to do the heavy lifting,\n``imphook`` just allows to plug it seamlessly into the Python import\nsystem):\n\n* Override importing of (all or some) .py files, to support new\n  syntax or semantics in them.\n* Import files written using a DSL (domain-specific language)\n  as if they were Python modules. E.g., config or data files.\n* Import modules written in other language(s), assuming you have\n  an interpreter(s) for them.\n* Import binary files, e.g. Java or LLVM bytecode.\n\n``imphook`` works both with new, lightweight legacy-free Python\nAPI, as promoted by the `Pycopy \u003chttps://github.com/pfalcon/pycopy\u003e`_\nPython dialect (the original source of the \"easy import hooks\" idea),\nand CPython (the older, reference Python implementation), and with\nother Python implementations which are CPython-compatible.\n\nQuick Start\n-----------\n\nMake sure that you already installed ``imphook`` using::\n\n    pip3 install -U imphook\n\nBelow is a complete example of an import hook module to load\n``key = value`` style config files::\n\n    import imphook\n\n    def hook(modname, filename):\n        with open(filename) as f:\n            # Create a module object which will be the result of import.\n            mod = type(imphook)(modname)\n            for l in f:\n                k, v = [x.strip() for x in l.split(\"=\", 1)]\n                setattr(mod, k, v)\n            return mod\n\n    imphook.add_import_hook(hook, (\".conf\",))\n\nSave this as the ``mod_conf.py`` file, and add the two following\nfiles to test it:\n\n``example_settings.conf``::\n\n    var1 = 123\n    var2 = hello\n\n``example_conf.py``::\n\n    import example_settings as settings\n\n    print(settings.var1)\n    print(settings.var2)\n\nNow run::\n\n    python3 -m imphook -i mod_conf example_conf.py\n\nAs you can see, the ``example_conf.py`` is able to import\n``example_settings.conf`` as if it were a normal Python module.\n\nBesides copy-pasting the above and other examples, you can also\nclone the Git repository of ``imphook``, which contains various\nready-to-use examples::\n\n    git clone https://github.com/pfalcon/python-imphook\n\n\nAPI to install hooks and hook structure\n---------------------------------------\n\nThe API of the module consists of one function:\n`imphook.add_import_hook(hook, ext_tuple)`. *hook* is a name of\nhook function. *ext_tuple* is a tuple of file extensions\nthe hook function should handle (the leading dot should be included).\nMore often than not, you will want to handle just one extension,\nso don't forget to use the usual Python syntax with a trailing\ncomma for 1-element tuple, e.g.: ``(\".ext\",)``. Python modules may\nnot contain a dot (``\".\"``) in their names (they are used to separate\nsubpackages), so the extension you register may contain multiple\ndots, e.g. ``\".foo.bar\"``, with filename ``my_module.foo.bar``\nmatching it.\n\nIt is possible to call `imphook.add_import_hook(hook, ext_tuple)`\nmultiple times to install multiple hooks. The hooks are installed\nin the stack-like fashion, the last installed will be called\nfirst. It is possible to install multiple hooks for the same file\nextension, and earlier installed hooks may still be called in this\ncase, because a hook function may skip processing a particular\nfile, and let other hooks to take a chance, with default processing\nhappening if no hook handled the import.\n\nThe signature and template of the actual hook function is::\n\n    def my_hook(modname, filename):\n        # Return None if you don't want to handle `filename`.\n        # Otherwise, load `filename`, create a Python module object,\n        # with name `modname`, populate it based on the loaded file\n        # contents, and return it.\n\nThe *modname* parameter is a full module name of the module to\nimport, in the usual dot-separated notation, e.g. ``my_module``\nor ``pkg.subp.mod``. For relative imports originated from within\na package, this name is already resolved to full absolute name.\nThe *modname* should be used to create a module object with the\ngiven name.\n\nThe *filename* parameter is a full pathname (with extension) of the\nfile which hook should import. This filename is known to exist, so\nyou may proceed to open it directly. You may skip processing this\nfile by returning ``None`` from the hook, then other hooks may be\ntried, and default processing happens otherwise (e.g. ``.py`` files\nare loaded as usual, or ImportError raised for non-standard\nextensions). For package imports, the value of *filename* ends with\n``/__init__.py``, and that is the way to distinguish module vs\npackage imports.\n\nIf the hook proceeds with the file, it should load it by whatever\nmeans suitable for the file type. File types which are not natively\nsupported by Python would require installing and using other extension\nmodules (beyond ``imphook``). After loading the file, the hook should\ncreate an empty Python module object which will be the result of the\nimport. There are a few ways to do that:\n\n* The baseline is to call a module type as a constructor. To get\n  a module type, just apply the usual ``type()`` function to an\n  existing (imported) module. You'll definitely have ``imphook``\n  itself imported, which leads us to::\n\n    mod = type(imphook)(modname)\n\n  The parameter to constructor is the name of module to create,\n  as passed to the hook.\n* If the above looks too magic for you, you can import symbolic\n  name for module type from the ``types`` module::\n\n    from types import ModuleType\n    mod = ModuleType(modname)\n\n* Finally, you may use the ``imp`` module, which may be as well\n  the clearest (to the newbie) way of doing it::\n\n    import imp\n    mod = imp.new_module(modname)\n\n  But mind that the ``imp`` module is considered deprecated.\n\nOf the choices above, the first is the most efficient - no need\nto import additional modules, and it's just one line. And once\nyou saw and were explained what it does, it shouldn't be a problem\nto remember and recognize it later.\n\nOnce the module object is created as discussed above, you should\npopulate it. A way to do that is by using ``setattr()`` builtin\nto set a particular attribute of a module to a particular value.\nAttributes usually represent variables with data values, but\nmay be also functions and classes.\n\nFinally, you just return the populated module object.\n\nIn case you want to perform custom transformation on the Python\nsource, the process is usually somewhat different, where you\ntransform a representation of the source, and then execute it\nin the context of a new module, which causes it to be populated.\nAn example of that is provided in the latter section.\n\n\nUsing import hooks in your applications\n---------------------------------------\n\nThere are 2 ways to use import hook(s) in you Python programs:\neither preloading them before starting your program using ``imphook``\ncommand-line runner (next section) or load them explicitly at the\nstartup of your application. Crucial thing to remember that import\nhooks apply: a) for imports only; b) for imports appearing after\nthe hook was installed.\n\nThe main file of our application is normally *not imported*, but\nexecuted directly. This leads to the following pattern in structuring\nyour application source files:\n\n* Have a \"startup file\", which is the one which user will actually\n  run, so name it appropriately. In that file, you load import hooks\n  and perform other baseline system-level initialization.\n* The main functionality of your application is contained in seperate\n  module(s). The startup script imports such a main module and\n  executes it (e.g., by calling a function from it).\n\nYou already grasped how that works - as the \"main\" module is\n*imported*, whatever hooks the \"startup\" script installed, will\napply to it.\n\nThis pattern is actually officially encoded in the structure of\nPython *packages*. And any non-trivial Python application will\nlikely be a package with a few sub-modules in it, so you can as\nwell structure your applications this way (as a package), even\nif they start simple (only one submodule initially). So, if you\ntry to \"run\" a package, what actually gets run is ``__main__``\nsubmodule in that package. That's exactly the \"startup\" file\nwe discussed above. It installs the import hooks, and imports\na submodule with actual application's functionality.\n\nThe actual loading of hooks is very easy: just import them in\nyour startup script, voila! For ``mod_conf.py`` hook module\nshown in the example above that would be::\n\n    import mod_conf\n\nYou should do that as soon as reasonably possible in your startup\nfile. Normally, that would be after stdlib imports, and before\nimports of your app's modules. Sometimes, you may want to put\nhook imports very first, even before the stdlib modules. E.g.,\nif hooks implement JIT compilation, which may benefit even stdlib\nmodules (someone yet has to develop such hooks!). All in all,\nfollow the guidelines above and documentation of the particular\nhooks that you use.\n\nFinally, the pattern described above (of having \"startup\" and\n\"main\" modules in your app) doesn't work too well in case your\napplication is a single script file, you would need to turn that\ninto 2 files to make the import hooks work. But that's exactly\nwhy ``imphook`` provides command-line preloader/runner interface!\n\n\nCommand-line interface\n----------------------\n\nWhere you would normally run a single script like:\n\n* ``python3 script.py``, or\n* ``python3 -m script``\n\nyou can run the same script/module with some import hooks preloaded\nusing following commands (changes comparing to the above commands\nare shown in italics):\n\n* ``python3`` *-m imphook -i \u003cmod_hook\u003e* ``script.py``, or\n* ``python3`` *-m imphook -i \u003cmod_hook\u003e* ``-m script``\n\nThat's exactly how we ran ``example_conf.py`` in the Quick Start\nsection. You can repeat ``-i`` option multiple times. Alternatively\nand more compactly, you can pass to single ``-i`` option a\ncomma-separated list of hook modules to import, e.g.:\n``-i mod_hook1,mod_hook2,mod_hook3``. If you pass multiple hooks,\nthe will be handled in the same stack-like fashion as the API\ncall described above. In the previous example, ``mod_hook3`` will\nbe called first to process imports, then ``mod_hook2``, then\n``mod_hook1``. Of course, this will be important only if more\nthan one hook handles the same file extenstion.\n\nThis stack-like order on the command-line is used for consistency\nwith the API, to avoid confusion between the two. But it's also\nthe natural order, if you think about it: we start with standard\nPython import hooks (yes, Python handles all imports using hooks,\nalthough its hooks are as simple and clear as those we build here\nwith ``imphook``). Then, there may be some hooks installed in\n``sitecustomize`` module (that's a way to install some \"persistent\"\nhooks for all your projects, which we don't go into, as it should\nbe known for any advanced user). When we get to the ``imphook``\ncommand line, we want to be able to override either standard\nPython or ``sitecustomize`` hooks, and that's why all hooks are\nconsistently installed in the stack-like fashion. And you should\nkeep in mind that if an application explicitly installs any hooks,\nthey will have higher priority than those passed on the command\nline.\n\nWe also should again emphasize the difference between ``script.py``\nand ``-m script`` forms above. In the first case, the script is\n*executed directly*, and any import hooks you specified with\n``-i`` **do not** apply to the script itself (but will apply to\nimports performed by ``script.py``). Even want hooks to apply\nto the script's source itself, you **must** run it using\n``-m script`` notation (which exactly tells \"import this script\nas a module, don't run it directly\"). Pay double attention that\nwhen you use ``-m`` switch, you **must not** use the ``.py``\nextension (if you do, you ask to import the submodule ``py`` of\npackage ``script``, which rarely what you want or makes sense).\n\nLast final note is about whitespace in the command-line parameters:\nthey should be used exactly as shown and described: there should\nalways be spaces between ``-i`` and ``-m`` options and their\nparameters. And vice versa, if you use comma-separated list of\nimport hooks, there should be no spaces in that list.\n\n\nExample of Python source transformation\n---------------------------------------\n\nWe started this documentation with a quick example of writing an\nimport hook for a simple DSL. We'd like to finish it with examples\nof another expected common use of ``imphook`` - implementing\nnew syntactic (and semantic!) features for Python.\n\nA common starting example is just trying to \"rename\" one of existing\nsyntactic elements, e.g. use word \"function\" instead of \"lambda\".\n\nSo, we'd like to get following source code dialect to run:\n\n``example_funkw.py``::\n\n    my_fun = function: print(\"imphook's functionality is cool!\")\n    my_fun()\n\nThe simplest way to do that is just to replace every occurance of\n\"function\" in the source with \"lambda\" (then compile, then execute\nit in the module context). We thus will come up with the following\nhook implementation:\n\n``mod_funkw_naive.py``::\n\n    import imphook\n\n    def hook(filename):\n        with open(filename) as f:\n            source = f.read()\n        source = source.replace(\"function\", \"lambda\")\n        mod = type(imphook)(\"\")\n        exec(source, vars(mod))\n        return mod\n\n    imphook.add_import_hook(hook, (\".py\",))\n\nIf you read previous sections carefully, you already know that if\nwe want the import hook to apply to the script itself, we must\nrun it as module, using ``-m`` switch::\n\n    python3 -m imphook -i mod_funkw_naive -m example_funkw\n\nAnd we get::\n\n    imphook's lambdaality is cool!\n\nOops! The word \"lambdaality\" is definitely cool, but that's not what\nwe expected! It happens because the code just blindly replaces\noccurrances everywhere, including within string literals. We could\ntry to work that around by using regular expression replace and match\nwhole words, that would help with the case above, but would still\nreplace lone \"function\" in the strings. Which makes us conclude:\ntransforming surface representation of a program (i.e. a sequence\nof characters) is never an adequte method. We should operate on\na more suitable program representation, and the baseline such\nrepresentation is a sequence (or stream) of tokens. Let's use it:\n\nmod_funkw.py::\n\n    import tokenize\n    import imphook\n\n    def hook(filename):\n\n        def xform(token_stream):\n            for t in token_stream:\n                if t[0] == tokenize.NAME and t[1] == \"function\":\n                    yield (tokenize.NAME, \"lambda\") + t[2:]\n                else:\n                    yield t\n\n        with open(filename, \"rb\") as f:\n            # Fairly speaking, tokenizing just to convert back to string form\n            # isn't too efficient, but CPython doesn't offer us a way to parse\n            # token stream so far, so we have no choice.\n            source = tokenize.untokenize(xform(tokenize.tokenize(f.readline)))\n        mod = type(imphook)(\"\")\n        exec(source, vars(mod))\n        return mod\n\n    imphook.add_import_hook(hook, (\".py\",))\n\n\nCredits and licensing\n---------------------\n\n``imphook`` is (c) `Paul Sokolovsky \u003chttps://github.com/pfalcon\u003e`_ and\nis released under the MIT license.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpfalcon%2Fpython-imphook","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpfalcon%2Fpython-imphook","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpfalcon%2Fpython-imphook/lists"}