How to print the output in during loop in ansible

Leave a Comment / General, scripting, tech

In ansible, we there are multiple ways to loop over a task. However, there are minimal ways to see what is happening in the background during the loop is getting executed. Sometimes, we want to display the loop output for each iteration to the user.

1. We use loop or with_items to iterate over the task; both loop and with_items expect a list as input to iterate over each element of the supplied list.
2. Similarly, we use until loop with retries and delay to rerun the task until a particular condition is met. 

We will see two options to see what's happening in the background during the loop execution and retries.
  1. The easiest out-of-the-box solution is to increase the verbosity to -vvv to see what's happening in the background.
  2. Writing our custom callback plugin to tweak the ansible's default behavior. We will focus on this option in this post.

Sample playbook

In the below playbook, we are doing the following:
- list the contents of /tmp/bar directory 6 times, every 1 second in the first task.
- Until the file /tmp/foo contains a string 'technekey' in the second task.
- The task will be retried 10 times, every 2 seconds in the second task.

---
- hosts: localhost
  gather_facts: false
  tasks:
  - name: "Print the contents of /tmp/bar/ directory"
    shell: ls -1 /tmp/bar/
    loop: [1,2,3,4,5,6]
    loop_control:
        pause: 1

  - name: "Check file writing status"
    shell: cat /tmp/foo
    register: file_contents
    until: file_contents.stdout|regex_search('technekey')
    retries: 10
    delay: 2

The default behavior

The following result will be produced when the user executes the above playbook.

ansible-playbook  example.yml


PLAY [localhost] ****************************************************************************************************************************************************

TASK [Print the contents of /tmp/bar/ directory] ********************************************************************************************************************
changed: [localhost] => (item=1)
changed: [localhost] => (item=2)
changed: [localhost] => (item=3)
changed: [localhost] => (item=4)
changed: [localhost] => (item=5)
changed: [localhost] => (item=6)

TASK [Check file writing status] ************************************************************************************************************************************
FAILED - RETRYING: [localhost]: Check file writing status (10 retries left).
FAILED - RETRYING: [localhost]: Check file writing status (9 retries left).
FAILED - RETRYING: [localhost]: Check file writing status (8 retries left).
FAILED - RETRYING: [localhost]: Check file writing status (7 retries left).
FAILED - RETRYING: [localhost]: Check file writing status (6 retries left).
FAILED - RETRYING: [localhost]: Check file writing status (5 retries left).
FAILED - RETRYING: [localhost]: Check file writing status (4 retries left).
FAILED - RETRYING: [localhost]: Check file writing status (3 retries left).
FAILED - RETRYING: [localhost]: Check file writing status (2 retries left).
FAILED - RETRYING: [localhost]: Check file writing status (1 retries left).
fatal: [localhost]: FAILED! => {"attempts": 10, "changed": true, "cmd": "cat /tmp/foo", "delta": "0:00:00.002239", "end": "2022-09-22 12:52:34.531610", "msg": "", "rc": 0, "start": "2022-09-22 12:52:34.529371", "stderr": "", "stderr_lines": [], "stdout": "dasdas", "stdout_lines": ["dasdas"]}

PLAY RECAP **********************************************************************************************************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Solutions

Solution-1: Use increased verbosity (-VVV),


you may choose to use -vvv to print the progress(the contents of each retry attempt). However, this will produce overwhelming output for all the tasks in the playbook. This may not be desirable for all cases.

ansible-playbook  example.yml -vvv
...
.....
FAILED - RETRYING: [localhost]: Check file writing status (10 retries left).Result was: {
    "attempts": 1,
    "changed": true,
    "cmd": "cat /tmp/foo",
    "delta": "0:00:00.002019",
    "end": "2022-09-21 11:15:04.850249",
    "invocation": {
        "module_args": {
            "_raw_params": "cat /tmp/foo",
            "_uses_shell": true,
            "argv": null,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "stdin": null,
            "stdin_add_newline": true,
            "strip_empty_ends": true,
            "warn": false
        }
    },
    "msg": "",
    "rc": 0,
    "retries": 11,
    "start": "2022-09-21 11:15:04.848230",
    "stderr": "",
    "stderr_lines": [],
    "stdout": "hello",
    "stdout_lines": [
        "hello"
    ]
}


Solution-2: Using a custom callback plugin


To use a custom callback plugin, you must create a directory called callback_plugins in the same directory as your playbook(example.yml). See the example below:

tree
.
├── callback_plugins
│   └── verbose_retry.py
└── example.yml

1 directory, 2 files

Alternatively, you can configure the location for your callback plugin in your ansible.cfg file

[defaults]
callback_plugins = ./path/to/your/callback/plugin/file.py   #<----any patch containing your callback code relative to ansible.cfg file.


Now, save the following code in callback_plugins/verbose_retry.py file. In this case, we are writing our login in the v2_runner_retry callback; based on your requirements, you can use different runners to write your code. See summary for URL for more examples.

from __future__ import (absolute_import, division, print_function)
__metaclass__ = type


from ansible.plugins.callback import CallbackBase
from ansible import constants as C
from ansible.utils.color import colorize, hostcolor

import subprocess
import datetime
import json

DOCUMENTATION = '''
    callback: verbose_retry
    type: stdout
    short_description:
      1. Print the stdout and stderr during until/delay/retries
      2. Print the results during the loop for each item
    description:
      - During the retry of any task, by default the task stdout and stderr are hidden from the user on the terminal.
      - To make things more transparent, we are printing the stdout and stderr to the terminal.
    requirements:
      - python subprocess,datetime,pprint
    '''

class CallbackModule(CallbackBase):

    '''
    Callback to various ansible runner calls.
    '''

    CALLBACK_VERSION = 2.0
    CALLBACK_TYPE = 'notification'
    CALLBACK_NAME = 'loop_and_retry_verbose'

    def __init__(self, *args, **kwargs):
        super(CallbackModule, self).__init__()


    #This code will be called when any iteration of loop/with_items is failed
    def v2_runner_item_on_failed(self, result):
        self._display.display(json.dumps(result._result,indent=2), color=C.COLOR_ERROR)

    
    #This code will be called when any iteration of loop/with_items is passed
    def v2_runner_item_on_ok(self, result):
        self._display.display(json.dumps(result._result,indent=2), color=C.COLOR_OK)


    #this code will be called when the until iteration is failed and retrying
    def v2_runner_retry(self, result):
        module_name = result._task_fields['args']['_ansible_module_name']
        self.play_host = result._host
        retries = result._result['retries']
        attempts = result._result['attempts']
        task_name = result.task_name or result._task
        # If the task has a variable set, verbose to true then only display in verbose
        try:
            p = subprocess.Popen(['stty', 'size'], stdout=subprocess.PIPE)
            columns = p.communicate()[0].split()[-1]
        except Exception as ex:
            columns = 80
        TITLE = '[' + str(datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"))+'] RETRYING(' + str(attempts) + '/' + str(retries-1) + '):' + task_name
        self._display.display(TITLE.ljust(int(columns), '*'))
        if len(result._result.get('stderr_lines')) == 0:
            self._display.display(json.dumps(result._result,indent=2), color=C.COLOR_OK)
        else:
            self._display.display(json.dumps(result._result,indent=2), color=C.COLOR_ERROR)

Sample output with callback module

ansible-playbook  example.yml


PLAY [localhost] ****************************************************************************************************************************************************

TASK [Print the contents of /tmp/bar/ directory] ********************************************************************************************************************
changed: [localhost] => (item=1)
{
  "changed": true,
  "stdout": "",
  "stderr": "",
  "rc": 0,
  "cmd": "ls -1 /tmp/bar/",
  "start": "2022-09-22 12:14:21.173535",
  "end": "2022-09-22 12:14:21.175448",
  "delta": "0:00:00.001913",
  "msg": "",
  "invocation": {
    "module_args": {
      "_raw_params": "ls -1 /tmp/bar/",
      "_uses_shell": true,
      "warn": false,
      "stdin_add_newline": true,
      "strip_empty_ends": true,
      "argv": null,
      "chdir": null,
      "executable": null,
      "creates": null,
      "removes": null,
      "stdin": null
    }
  },
  "stdout_lines": [],
  "stderr_lines": [],
  "_ansible_no_log": false,
  "item": 1,
  "ansible_loop_var": "item",
  "_ansible_item_label": 1
}
changed: [localhost] => (item=2)
{
  "changed": true,
  "stdout": "",
  "stderr": "",
  "rc": 0,
  "cmd": "ls -1 /tmp/bar/",
  "start": "2022-09-22 12:14:24.288338",
  "end": "2022-09-22 12:14:24.290434",
  "delta": "0:00:00.002096",
  "msg": "",
  "invocation": {
    "module_args": {
      "_raw_params": "ls -1 /tmp/bar/",
      "_uses_shell": true,
      "warn": false,
      "stdin_add_newline": true,
      "strip_empty_ends": true,
      "argv": null,
      "chdir": null,
      "executable": null,
      "creates": null,
      "removes": null,
      "stdin": null
    }
  },
  "stdout_lines": [],
  "stderr_lines": [],
  "_ansible_no_log": false,
  "item": 2,
  "ansible_loop_var": "item",
  "_ansible_item_label": 2
}
..................
.........................
............................
.................................
TASK [Check file writing status] ************************************************************************************************************************************
FAILED - RETRYING: [localhost]: Check file writing status (10 retries left).
[2022-09-22 12:16:26] RETRYING(1/10):Check file writing status*******************************************************************************************************
{
  "changed": true,
  "stdout": "dasdas",
  "stderr": "",
  "rc": 0,
  "cmd": "cat /tmp/foo",
  "start": "2022-09-22 12:16:26.554664",
  "end": "2022-09-22 12:16:26.556560",
  "delta": "0:00:00.001896",
  "msg": "",
  "invocation": {
    "module_args": {
      "_raw_params": "cat /tmp/foo",
      "_uses_shell": true,
      "warn": false,
      "stdin_add_newline": true,
      "strip_empty_ends": true,
      "argv": null,
      "chdir": null,
      "executable": null,
      "creates": null,
      "removes": null,
      "stdin": null
    }
  },
  "stdout_lines": [
    "dasdas"
  ],
  "stderr_lines": [],
  "_ansible_no_log": false,
  "attempts": 1,
  "retries": 11
}
FAILED - RETRYING: [localhost]: Check file writing status (9 retries left).
[2022-09-22 12:16:28] RETRYING(2/10):Check file writing status*******************************************************************************************************
{
  "changed": true,
  "stdout": "dasdas",
  "stderr": "",
  "rc": 0,
  "cmd": "cat /tmp/foo",
  "start": "2022-09-22 12:16:28.677563",
  "end": "2022-09-22 12:16:28.679569",
  "delta": "0:00:00.002006",
  "msg": "",
  "invocation": {
    "module_args": {
      "_raw_params": "cat /tmp/foo",
      "_uses_shell": true,
      "warn": false,
      "stdin_add_newline": true,
      "strip_empty_ends": true,
      "argv": null,
      "chdir": null,
      "executable": null,
      "creates": null,
      "removes": null,
      "stdin": null
    }
  },
  "stdout_lines": [
    "dasdas"
  ],
  "stderr_lines": [],
  "_ansible_no_log": false,
  "attempts": 2,
  "retries": 11
}
FAILED - RETRYING: [localhost]: Check file writing status (8 retries 
........................
...........................
...............................
......................................

Summary:


Using callback plugins, we can make the playbooks spit the desired output as per our requirement and make the playbooks incredibly powerful. You can get tons of references using the following URLs:

1. https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/callback/default.py
2. https://docs.ansible.com/ansible/latest/plugins/callback.html

Related Posts

0 0 votes
Please, Rate this post
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x
Scroll to Top