Why Ansible assert does not fail when variable is not defined inside a loop?

Question

I use the test: variable is defined to assert in a play if a variable is defined, and if it's not empty.

But the test does not fail when used on an undefined variable, and the module responds All assertions passed!

The result is not what I expect. I expect that non existing variables will trigger a false assertion.

My playbook assertpb.yml is:

- name: Playbook to try assert module
  hosts: all
  gather_facts: false

  tasks:
    - name: Check mandatory variableA
      assert:
        quiet: false
        that:
          - variableA is defined
          - variableA | length > 0

    - name: Check mandatory variableB
      assert:
        quiet: false
        that:
          - variableB is defined
          - variableB | length > 0

    - name: Check mandatory variables with loop
      assert:
        quiet: false
        that:
          - item is defined
          - item | length > 0
      loop:
        - variableA
        - variableB
        - variableThatDoesNotExist

    - meta: end_play

Result:

$ ansible-playbook -i inventory -l testmachine assertpb.yml -e variableA=coucou -e variableB=bisous

PLAY [Playbook to try assert module] **********************************************************************************************************************************************************

TASK [Check mandatory variableA] **************************************************************************************************************************************************************
ok: [testmachine] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [Check mandatory variableB] **************************************************************************************************************************************************************
ok: [testmachine] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [Check mandatory variables with loop] ****************************************************************************************************************************************************
ok: [testmachine] => (item=variableA) => {
    "ansible_loop_var": "item",
    "changed": false,
    "item": "variableA",
    "msg": "All assertions passed"
}
ok: [testmachine] => (item=variableB) => {
    "ansible_loop_var": "item",
    "changed": false,
    "item": "variableB",
    "msg": "All assertions passed"
}
ok: [testmachine] => (item=variableThatDoesNotExist) => {
    "ansible_loop_var": "item",
    "changed": false,
    "item": "variableThatDoesNotExist",
    "msg": "All assertions passed"
}

TASK [meta] ***********************************************************************************************************************************************************************************

PLAY RECAP ************************************************************************************************************************************************************************************
testmachine : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

What's wrong here?

• In the playbook?
• In the test used in that: parameter?
• In the loop construct?

Answer 1

When your assertion is inside the that parameter of an assertion task, the information it contains is templated:

A list of string expressions of the same form that can be passed to the when statement.

^{From the documentation of the that parameter of the assert module.}

and

The when clause is a raw Jinja2 expression without double curly braces

^{From the documentation page "Basic conditionals with when".}

But, when you do use it as the element of a loop, the loop does treat the list given as a simple YAML, it does not automatically templates the variables for you!
So, those item's end up being literal strings and those strings are of course defined and not zero-sized.

So, what you really meant in you loop was:

loop:
  - "{{ variableA }}"
  - "{{ variableB }}"
  - "{{ variableThatDoesNotExist }}"

---

So, with

- name: Check mandatory variables with loop
  assert:
    quiet: false
    that:
      - item is defined
      - item | length > 0
  loop:
    - "{{ variableA }}"
    - "{{ variableB }}"
    - "{{ variableThatDoesNotExist }}"

It, then, raises your expected:

TASK [Check mandatory variables with loop] *********************************
fatal: [localhost]: FAILED! => 
  msg: '''variableThatDoesNotExist'' is undefined'

---

And if you want the variables to assert out of a list of literal strings, use the vars lookup.
So, with:

- name: Check mandatory variables with loop
  assert:
    quiet: false
    that:
      - lookup('vars', item) is defined
      - lookup('vars', item) | length > 0
  loop:
    - variableA
    - variableB
    - variableThatDoesNotExist

You would also get your expected result:

TASK [Check mandatory variables with loop] *************************************************************************************
ok: [localhost] => (item=variableA) => changed=false 
  ansible_loop_var: item
  item: variableA
  msg: All assertions passed
ok: [localhost] => (item=variableB) => changed=false 
  ansible_loop_var: item
  item: variableB
  msg: All assertions passed
fatal: [localhost]: FAILED! => 
  msg: 'The conditional check ''lookup(''vars'', item) is defined'' failed. The error was: error while evaluating conditional (lookup(''vars'', item) is defined): No variable found with this name: variableThatDoesNotExist'