A deep dive

In Python, an import hook has two main components:

  • A Finder which looks for the code (usually a .py file) that is requested by an import statement

  • A Loader which retrieves the source code, executes it, an returns a module object.

The order of execution is the following

Import hook overview

The diagram above illustrates only the main steps. These can be broken down further as follows.

Import hook details

Using Ideas

Normally, for creating import hooks and as shown above, it is important to distinguish the two main phases, that is creating a Finder and a Loader. Using ideas, these are automatically done for us, and we can focus on various parts over which we can have control.

In the diagram below:

  • Inside each of the major blocks (Decode, AST, Bytecode), we don’t have control over the individual components; however, we could, in principle, substitute our own version of the entire block.

  • There exists at least one example for anything (excluding major blocks) with a white background.

  • Anything with a light blue background indicates that some examples of this should be doable. Ideally, at least one example of each possible case should be added.

ideas import hook possibilities

Options to create a custom hook

ideas.import_hook.create_hook(callback_params=None, create_module=None, console_dict=None, exec_=None, extensions=None, first=True, hook_name=None, ipython_ast_node_transformer=None, module_class=None, source_init=None, transform_ast=None, transform_bytecode=None, transform_source=None)[source]

Function to facilitate the creation of an import hook.

Each of the following parameter is optional; most of these are never needed except in some unusual import hooks.

Usually, at least one of transform_ast, transform_bytecode, and transform_source should be specified.

  • callback_params: a dict containing keyword parameters to be passed back to the transform_source function.

  • create_module: a custom function to create a module object instead of using Python’s default.

  • console_dict: a dict object used as ‘locals’ with the Ideas console, instead of its usual default.

  • exec_: a custom method used to execute the source code inside a module’s dict.

  • extensions: a list of file extensions, other than the usual .py, etc., used to identify modules containing source code.

  • first: if True, the custom hook will be used as the first location in sys.meta_path, to look for source files.

  • hook_name: used to give a more readable repr to the hook created.

  • ipython_ast_node_transformer: used to do AST transformations in an IPython/Jupyter environment. It should be a class derived from ast.NodeTransformer and return a node.

  • module_class: custom class to use for the module created instead of the default one assigned by Python.

  • source_init: custom code to be executed before any code from a user is executed. For example, if one creates an import hook that treats every float as a Decimal object, this custom code could be:

    from decimal import Decimal

  • transform_ast: used to do AST transformations in a Python environment (excluding IPython/Jupyter). It should be a class derived from ast.NodeTransformer, eventually returning a tree object.

  • transform_bytecode: used to mutate a code object.

  • transform_source: used to transform some source code prior to execution.

About Decode

In the last diagram shown above, there is a block labeled ‘Decode’. Changing the way that Python processes code during this phase does not require the creation of an import hook; instead, it requires the use of a custom codec.

An example is shown in a later section.