Writing Custom module in ansible

Ansible ships with a massive collection of built-in modules; these modules are pretty good in doing their job, cover most of the scenarios, have good documentation, and have a very reliable community backing them. If you wish to see the list or count of the inbuilt modules ansible provides, you can run the ansible-doc -l 2>/dev/null command to get the list. You must be amazed by the number of modules already shipped with Ansible. However, if you have worked around ansible for some time, you might have realized that ansible does not have modules to do everything. Sometimes, we want to do something that is not possible using plain ansible; thankfully, ansible is extensible.


To perform a very custom or complex task, we can either write a script and call it via script module or write our module with whatever business logic we want. It depends on the scenario to decide which solution is better for a case. Sometimes scripts could be handy for a fast turnaround. However, there are a few downsides to using scripts. A custom module could be very much like a script but more tightly coupled with ansible. Writing the custom module is easy; in this post, we will write our custom module. Python is the most commonly used language for custom modules; we can write custom modules using any language that returns JSON to ansible. However, we will write our custom module in python.

I have also written a post about an ansible custom module using a shell script. You can also read about it at this URL.

Why not just use a script? In some cases, it’s perhaps a good idea.

I think you can use the script for the cases where you rarely need to perform that task; however, when you need to repeat such task multiple times with different arguments, then perhaps it’s better to do it in an ansible native way. By doing the “module” way, you will natively have a uniform playbook structure. Also, ansible documentation provides a few reasons for using a module over a script; you can run the “ansible-doc script” command to see the same info as below.

<sub>NOTES:
* It is usually preferable to write Ansible modules rather than pushing scripts. Convert your script to an Ansible module for bonus points!
* The ssh' connection plugin will force pseudo-tty allocation via-tt' when scripts are executed. Pseudo-ttys do not have a stderr channel and all stderr is sent to stdout. If you depend on separated stdout and stderr result keys, please switch to a copy+command set of tasks instead of using script.
* If the path to the local script contains spaces, it needs to be quoted.</sub>

If you do not trust me, call the following script from your playbook using the “script module“

#name of the file is my_script.sh
#!/bin/bash

echo "This line is written in a script"
echo "This line is also written in the script, but on stderr" >&2

---
- hosts: webservers
  gather_facts: False
  tasks:
  - name: "Calling my script"
    script: my_script.sh
    register: script_output

  - debug: var=script_output

The above playbook would return something like the below; note that the stdout also has stderr merged. To make things worse, this behavior is only seen when the script module is executed over SSH, so if the script is called on localhost, it will exhibit different behavior than when executed on the remote host.

ok: [192.168.122.152] => {
    "script_output": {
        "changed": true,
        "failed": false,
        "rc": 0,
        "stderr": "Shared connection to 192.168.122.152 closed.\r\n",
        "stderr_lines": [
            "Shared connection to 192.168.122.152 closed."
        ],
        "stdout": "This line is written in a script\r\nThis line is also written in the script, but on stderr\r\n",
        "stdout_lines": [
            "This line is written in a script",
            "This line is also written in the script, but on stderr"
        ]
    }
}

Example/imaginary use case for writing a custom module

Write a module that would return area or volume Of an object, the module should take the following items as input arguments:
- length [expect int]
- breath [expect int]
- height [expect int]
- action [expect str<area or volume>]

NOTE: ^{_{I know you do not need a new module to calculate the area or volume of a rectangle. This example is chosen for the ease of showcasing an example.}}

Step-1: Create the playbook, and copy the below content in example.yml

In the below example playbook called example.yml, we have used a module called "foobar", this module is obviously not inbuilt or exists yet; we will create it later in this post. If you execute this playbook now, this will fail as the module called foobar does not exist yet.

---
- hosts: localhost
  gather_facts: False
  tasks:
  - name: "Get the area"
    foobar:
      length: 10
      breath: 8
      height: 9
      action: 'area'
    register: area_out

  - name: "Print the area "
    debug:
      msg: "{{ area_out.stdout }}"

  - name: "Get the volume"
    foobar:
      length: 10
      breath: 8
      height: 9
      action: 'volume'
    register: vol_out

  - name: "Print the volume"
    debug:
      msg: "{{ vol_out.stdout }}"

Step-2: The directory structure, location of the module file(location is important)

When the ansible-playbook is executed, it automatically looks at the paths defined under the ‘DEFAULT_MODULE_PATH‘ variable for the presence of required inbuilt modules. You may run ansible-config dump|grep DEFAULT_MODULE_PATH a command to get the list of paths where ansible search for the module.

In the case of custom modules, ansible will start searching for the code at 'DEFAULT_MODULE_PATH'; once the code for the custom module is not found at ‘DEFAULT_MODULE_PATH‘, then the ansible looks for the modules in a directory called 'library' at the same directory as the playbook. For example, if my playbook name is example.py I will create a directory called 'library‘ at the same level as my playbook and a module file called.’library/foobar.py‘

Run the following command to create the library directory:

mkdir library

#create a blank module file
touch library/foobar.py

##expected directory and file structure

technekey@controller:~/custom_modules$ tree
.
├── example.yml
├── library
│   └── foobar.py
1 directories, 2 files

Step-3: Create the module file with your business logic

#!/usr/bin/python3


from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
from ansible.module_utils.basic import AnsibleModule


#example of a few local functions, to calculate area or volume
#you can have your business logic here in form of functions

def get_area(length, breath):
  return length * breath

def get_volume(length, breath, height):
  return length * breath * height

#
# This is the main function body 
#
def main():
    # Define the args required by the module
    # update the aruments as per your requirements
    module_args = dict(
        length=dict(type='int', required=True),
        breath=dict(type='int', required=True),
        height=dict(type='int', required=True),
        action=dict(type='str', required=True),
        radius=dict(type='str', required=False),
    )
    #AnsibleModule class is imported from ansible.module_utils.basic
    # following is the allowed args to AnsibleModule
    # classansible.module_utils.basic.AnsibleModule(argument_spec, bypass_checks=False, no_log=False, mutually_exclusive=None, required_together=None, required_one_of=None, add_file_common_args=False, supports_check_mode=False, required_if=None, required_by=None)
    
    module = AnsibleModule(
        argument_spec=module_args,
        supports_check_mode=True
    )
    #build the default structure of return value, with all empty values
    #we will update this dict called 'result' while returning form the
    #the module with appropriate values

    result = dict(
        msg = '',
        stdout = '',
        stdout_lines = [],
        stderr = '',
        stderr_lines = [],
        rc = 0,
        failed = False,
        changed=False
    )
    
    #validate that the action argument is set to one of the allowed values
    allowed_actions = ['area', 'volume']
    if module.params['action'] not in allowed_actions:
        result['failed'] = True
        result['stderr'] = "Invalid argument supplied to this module"
        module.exit_json(**result)
    action =  module.params['action']
    length =  module.params['length']
    breath =  module.params['breath']
    height =  module.params['height']
    if action == 'area':
        area = get_area(length, breath)
        result['stdout'] = area
        module.exit_json(**result)
    if action == 'volume':
        volume = get_volume(length, breath, height)
        result['stdout'] = volume
        module.exit_json(**result)
    module.exit_json(**result)

if __name__ == '__main__':
    main()

Step-4: Execute the playbook that uses the foobar module

ansible-playbook example.yml


PLAY [localhost] ****************************************************************************************************************************************************

TASK [Get the area] *************************************************************************************************************************************************
ok: [localhost]

TASK [Print the area] ***********************************************************************************************************************************************
ok: [localhost] => {
    "msg": "80"
}

TASK [Get the volume] ***********************************************************************************************************************************************
ok: [localhost]

TASK [Print the volume] *********************************************************************************************************************************************
ok: [localhost] => {
    "msg": "720"
}

PLAY RECAP **********************************************************************************************************************************************************
localhost                  : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Step-5: Using classes with shared functions{for advance usage only}

As we can see that, a new module called ‘foobar‘ is working and returning the desired result. This indicates that ansible is successfully able to find the module file in the library directory automatically. If you are okay with writing all the functions and logic in the same module file (foobar.py), then you are good to go! You do not need to read further.

However, if you want to keep your code organized by keeping all your functions in a separate file, then read further.
We create a class and keep standard functions in that class; the reason for doing this is to keep our module file clean and small and, most importantly, to reuse the common functions among different modules.

Step-5A: Create a module_utils directory at the same level as the library directory

In this step, we will create a class and put all the functions that may be shared with other python modules. By doing this, we will keep the module file clutter free and reuse the code. Here is one crucial point, the class file must be created in a directory called “module_utils.”

technekey@controller:~/custom_modules$ mkdir module_utils/

#expected directory and file structure
technekey@controller:~/custom_modules$ tree 
.
├── example.yml
├── library
│   └── foobar.py
└── module_utils
    

2 directories, 2 files

Step-5B: Create the class file where we will put all of the common functions

technekey@controller:~/custom_modules$ touch module_utils/my_additional_utils.py
#expected directory and file structure
technekey@controller:~/custom_modules$ tree .
.
├── example.yml
├── library
│   └── foobar.py
└── module_utils
    └── my_additional_utils.py

2 directories, 3 files

Step-5C: Put all your business logic/shared functions inside module_utils

Notice that I have moved get_area and get_volume functions to the module_utils/my_additional_utils.py file. Now, these functions are not present in the main module file(foobar.py).

Essentially here, we have created a class called “my_functions” and created all the functions(mainly shared ones) in this class.

technekey@controller:~/custom_modules$ cat module_utils/my_additional_utils.py
#in this file, write common functions that you think might be usable for other modules
import math

class my_functions:
    def get_circle_area(radius):
        return f"{math.pi * radius * radius}"

    def get_area(length, breath):
        return length * breath

    def get_volume(length, breath, height):
        return length * breath * height

Step-5d: update the module file with no local function body

In the following example, note the additional import statement; Here, we are importing the class with all my functions.

from ansible.module_utils.my_additional_utils import my_functions

Now, the get_area and get_volume functions are referenced differently in the below file. For example, like my_functions.get_area and my_functions.get_volume.

#!/usr/bin/python3

from __future__ import (absolute_import, division, print_function)
__metaclass__ = type

from ansible.module_utils.basic import AnsibleModule

#here we are importing all the functions in the class we wrote earlier.
from ansible.module_utils.my_additional_utils import my_functions


def main():
    # Define the args required by the module
    module_args = dict(
        length=dict(type='int', required=True),
        breath=dict(type='int', required=True),
        height=dict(type='int', required=True),
        action=dict(type='str', required=True),
        radius=dict(type='str', required=False),
    )
    module = AnsibleModule(
        argument_spec=module_args,
        supports_check_mode=True
    )
    #build the default structure of return value, with all empty values
    #we will update this dict called 'result' while returning form the
    #the module with appropriate values

    result = dict(
        msg = '',
        stdout = '',
        stdout_lines = [],
        stderr = '',
        stderr_lines = [],
        rc = 0,
        failed = False,
        changed=False
    )
    allowed_actions = ['area', 'volume']
    if module.params['action'] not in allowed_actions:
        result['failed'] = True
        result['stderr'] = "Invalid argument supplied to this module"
        module.exit_json(**result)
    action =  module.params['action']
    length =  module.params['length']
    breath =  module.params['breath']
    height =  module.params['height']
    if action == 'area':
        area = my_functions.get_area(length, breath)
        result['stdout'] = area
        module.exit_json(**result)
    if action == 'volume':
        volume = my_functions.get_volume(length, breath, height)
        result['stdout'] = volume
        module.exit_json(**result)
    module.exit_json(**result)

if __name__ == '__main__':
    main()

What if you do not wish to create a library or module_utils directories?

Both ‘library' and ‘module_utils‘ directories are configurable; you can use environment variables or ansible.cfg to configure their path. This will make it ansible to search for modules in the configured directories. See this for module_utils configuration and this for library configuration. A sample example is as follows:

[defaults]
library = /path/to/my/module/files
module_utils = /path/to/my/module_utils

Summary:

If your module is doing some weighty, heavy lifting and sharing code with other modules, then only step 5 is required. For more straightforward use cases, step 5 is perhaps overkill.
Having said that, the ability to write our logic in the form of a module makes ansible super powerful.