Advanced library usage

Docker-Map can be used within Python applications directly, but also be used as a base implementation for other libraries. This section covers some areas that may be relevant when implementing enhancements, like Docker-Fabric.

Implementing policies

Before version 0.7.0, policies compared container maps to the current state on the Docker client, and performed changes directly. In later versions, implementations of BasePolicy only define a few guidelines, such as how containers are named, how image names are resolved, and which client objects to use:

Changing behavior

Operations are performed by a set of three components:

  • So-called state generators, implementations of AbstractStateGenerator, determine the current status of a container. They also establish if and in which the dependency path is being followed. Currently there are four implementations:
    • SingleStateGenerator detects the basic state of a single container configuration, e.g. existence, running, exit code.
    • DependencyStateGenerator is an extension of the aforementioned and used for forward-directed actions such as creating and starting containers, running scripts etc. It follows the dependency path of a container configuration, i.e. detecting states of a dependency first.
    • UpdateStateGenerator is a more sophisticated implementation of DependencyStateGenerator. In addition to the basic state it also checks for inconsistencies between virtual filesystems shared between containers and differences to the configuration.
    • DependentStateGenerator also detects the basic state of containers, but follows the reverse dependency path and is therefore used for stopping and removing containers.
  • Action generators, implementations of AbstractActionGenerator, transform these states into planned client actions. There is one action generator implementation, e.g. CreateActionGenerator aims to create all containers along the detected states that do not exist.
  • The runners perform the planned actions the client. They are implementations of AbstractRunner and decide how to direct the client to applying the container configuration, i.e. which methods and arguments to use. Currently there is only one implementation: DockerClientRunner.

The instance of MappingDockerClient decides which elements to use. For each action a pair of a state generator and action generator is configured in generators. runner_class defines which runner implementation to use.

Lazy resolution of variables

Container maps can be modified at any time, but sometimes it may be more practical to defer the initialization of variables to a later point. For example, if you have a function get_path(arg1, keyword_arg1='kw1', keyword_arg2='kw2'), you would usually assign the result directly:

container_map.host.volume1 = get_path(arg1, keyword_arg1='kw1', keyword_arg2='kw2')

If the value is potentially not ready at the time the container map is being built, the function call can be delayed until volume1 is actually used by a container configuration. In order to set a value for lazy resolution, wrap the function and its arguments inside dockermap.functional.lazy or dockermap.functional.lazy_once. The difference between the two is that the latter stores the result and re-uses it whenever it is accessed more than once, while the former calls the function and reproduces the current value on every use:

from dockermap.functional import lazy
container_map.host.volume1 = lazy(get_path, arg1, keyword_arg1='kw1', keyword_arg2='kw2')

or:

from dockermap.functional import lazy_once
container_map.host.volume1 = lazy_once(get_path, arg1, keyword_arg1='kw1', keyword_arg2='kw2')

Serialization issues

In case of serialization, it may not be possible to customize the behavior using aforementioned lazy functions. Provided that the input values can be represented by serializable Python types, these types can be registered for pre-processing using register_type().

For example, if a library uses MsgPack for serializing data, you can represent a value for serialization with:

from msgpack import ExtType

MY_EXT_TYPE_CODE = 1
...
container_map.host.volume1 = ExtType(MY_EXT_TYPE_CODE, b'info represented as bytes')

ExtType is supported by MsgPack’s Python implementation, and therefore as long as the byte data carries all information necessary to reproduce the actual value, no additional steps are necessary for serialization. During deserialization, you could usually reconstruct your original value by writing a simple function and passing this in ext_hook:

def my_ext_hook(code, data):
    if code == MY_EXT_TYPE_CODE:
        # This function should reconstruct the necessary information from the serialized data.
        return my_info(data)
    return ExtType(code, data)

This is the preferred method. If you however do not have access to the loading function (e.g. because it is embedded in another library you are using), you can slightly modify aforementioned function, and register ExtType for late value resolution:

from dockermap.functional import register_type

def my_ext_hook(ext_data):
    if ext_data.code == MY_EXT_TYPE_CODE:
        return my_info(ext_data.data)
    raise ValueError("Unexpected ext type code.", ext_data.code)

register_type(ExtType, my_ext_hook)

Note that you have to register the exact type, not a superclass of it, in order for the lookup to work.

Pre-resolving values

Aforementioned type registry is limited to values as listed in Availability. Additionally it may be difficult to detect errors in the configuration beforehand. In case the data can be pre-processed at a better time (e.g. after deserialization, in a configuration method), the method dockermap.funcitonal.resolve_deep() can resolve a structure of lists and dictionaries into their current values.

Rather than registering types permanently, they can also be passed to that function for temporary use, e.g.:

from dockermap.functional import expand_type_name, resolve_deep

# assume aforementioned example of my_ext_hook

resolve_dict = {expand_type_name(ExtType): my_ext_hook}
map_content = resolve_deep(deserialized_map_content, types=resolve_dict)

Availability

Lazy value resolution is available at the following points:

  • On container maps:
  • Within container configurations:
    • the user property;
    • host ports provided in the exposes, but not for the exposed port of the container (i.e. the first item of the tuple);
    • elements of create_options and start_options;
    • items of binds, if they are not volume aliases, i.e. they directly describe container volume and host path.
    • command line and user defined in each element of exec_commands;
    • elements listed in shares;
    • and on the network endpoint configurations in networks.
  • On network configurations:
    • the values of driver_options,
    • and the values of create_options.
  • On client configuration: For addresses in interfaces.