Welcome to part 2 of my Jinja2 Tutorial. In part 1 we learned what Jinja2 is, what are its uses, and we started looking at templating basics. Coming up next are loops and conditionals, sprinkled with tests and a healthy dose of examples!

Jinja2 Tutorial series

Contents

Control structures

In Jinja2 loops and conditionals come under name of control structures, since they affect flow of a program. Control structures use blocks enclosed by {% and %} characters.

Loops

First of the structures we'll look at is loops.

Jinja2 being a templating language has no need for wide choice of loop types so we only get for loop.

For loops start with {% for my_item in my_collection %} and end with {% endfor %}. This is very similar to how you'd loop over an iterable in Python.

Here my_item is a loop variable that will be taking values as we go over the elements. And my_collection is the name of the variable holding reference to the iterated collection.

Inside of the body of the loop we can use variable my_item in other control structures, like if conditional, or simply display it using {{ my_item }} statement.

Ok, but where would you use loops you ask? Using individual variables in your templates works fine for the most part but you might find that introducing hierarchy, and loops, will help with abstracting your data model.

For instance, prefix lists or ACLs are composed of a number of lines. It wouldn't make sense to have these lines represented as individual variables.

Initially you could model a specific prefix list using one variable per line, like so:

PL_AS_65003_IN_line1: "permit 10.96.0.0/24"
PL_AS_65003_IN_line2: "permit 10.97.11.0/24"
PL_AS_65003_IN_line3: "permit 10.99.15.0/24"
PL_AS_65003_IN_line4: "permit 10.100.5.0/25"
PL_AS_65003_IN_line5: "permit 10.100.6.128/25"

Which could be used in the following template:

# Configuring Prefix List
ip prefix-list PL_AS_65003_IN
 {{ PL_AS_65003_IN_line1 }}
 {{ PL_AS_65003_IN_line2 }}
 {{ PL_AS_65003_IN_line3 }}
 {{ PL_AS_65003_IN_line4 }}
 {{ PL_AS_65003_IN_line5 }}

Rendering results:

# Configuring Prefix List
ip prefix-list PL_AS_65003_IN
 permit 10.96.0.0/24
 permit 10.97.11.0/24
 permit 10.99.15.0/24
 permit 10.100.5.0/25
 permit 10.100.6.128/25

This approach, while it works, has a few problems.

If we wanted to have more lines in our prefix list we'd have to create another variable, and then another one, and so on. We not only have to add these new items to our data structure, templates would also have to have all of these new variables included individually. This is not maintainable, consumes a lot of time and is very error prone.

There is a better way, consider the below data structure:

PL_AS_65003_IN:
  - permit 10.96.0.0/24
  - permit 10.97.11.0/24
  - permit 10.99.15.0/24
  - permit 10.100.5.0/25
  - permit 10.100.6.128/25

And the template rendering prefix list configuration:

# Configuring Prefix List
ip prefix-list PL_AS_65003_IN
{%- for line in PL_AS_65003_IN %}
 {{ line -}}
{% endfor %}

After rendering:

# Configuring Prefix List
ip prefix-list PL_AS_65003_IN
 permit 10.96.0.0/24
 permit 10.97.11.0/24
 permit 10.99.15.0/24
 permit 10.100.5.0/25
 permit 10.100.6.128/25

If you look closely you'll notice this is essentially modeling the same thing, a prefix list with a number of entries. But by using list we clearly state our intent. Even visually you can tell straight away that all of the indented lines belong to the PL_AS_65003_IN.

Adding to the prefix list here is simple, we just need to append a new line to the block. Also, our templates don't have to change at all. If we used loop to iterate, like we did here, over this list then the new lines will be picked up if we re-run the rendering. Small change but makes things a lot easier.

You might have noticed that there's still room for improvement here. Name of the prefix list is hardcoded in the prefix list definition and in our for loop. Fear not, that's something we'll be improving upon shortly.

Looping over dictionaries

Let's now see how we can loop over dictionaries. We will again use for loop construct, remember, that's all we've got!

We can use the same syntax we used for iterating over elements of the list but here we'll iterate over dictionary keys. To retrieve value assigned to the key we need to use subscript, i.e. [], notation.

One advantage of using dictionaries over lists is that we can use names of elements as a reference, this makes retrieving objects and their values much easier.

Say we used list to represent our collection of interfaces:

interfaces:
  - Ethernet1:
      description: leaf01-eth51
      ipv4_address: 10.50.0.0/31
  - Ethernet2:
      description: leaf02-eth51
      ipv4_address: 10.50.0.2/31

There is no easy way of retrieving just Ethernet2 entry. We would either have to iterate over all elements and do key name comparison or we'd have to resort to advanced filters.

One thing to note, and this is hopefully becoming apparent, is that we need to spend some time modeling our data so that it's easy to work with. This is something you will rarely get right on your first attempt so don't be afraid to experiment and iterate.

Following with our example, we can keep data on individual interfaces assigned to keys in interfaces dictionary, instead of having them in a list:

interfaces:
  Ethernet1:
    description: leaf01-eth51
    ipv4_address: 10.50.0.0/31
  Ethernet2:
    description: leaf02-eth51
    ipv4_address: 10.50.0.2/31

Now we can access this data in our template like so:

{% for intf in interfaces -%}
interface {{ intf }}
 description {{ interfaces[intf].description }}
 ip address {{ interfaces[intf].ipv4_address }}
{% endfor %}

Giving us end result:

interface Ethernet1
 description leaf01-eth51
 ip address 10.50.0.0/31
interface Ethernet2
 description leaf02-eth51
 ip address 10.50.0.2/31

Here intf refers to Ethernet1 and Ethernet2 keys. To access attributes of each interface we need to use interfaces[intf] notation.

There is another way of iterating over dictionary, which I personally prefer. We can retrieve key and its value at the same time by using items() method.

{% for iname, idata in interfaces.items() -%}
interface {{ iname }}
 description {{ idata.description }}
 ip address {{ idata.ipv4_address }}
{% endfor %}

The end result is the same but by using items() method we simplify access to the attributes. This becomes especially important if you want to recursively iterate over deeply nested dictionaries.

I also promised to show how prefix list example can be improved upon, and that's where items() comes in.

We make small modification to our data structure by making each prefix list name a key int the dictionary prefix_lists

prefix_lists:
  PL_AS_65003_IN:
    - permit 10.96.0.0/24
    - permit 10.97.11.0/24
    - permit 10.99.15.0/24
    - permit 10.100.5.0/25
    - permit 10.100.6.128/25

We now add outer loop iterating over key, value pairs in dictionary:

# Configuring Prefix List
{% for pl_name, pl_lines in prefix_lists.items() -%}
ip prefix-list {{ pl_name }}
{%- for line in pl_lines %}
 {{ line -}}
{%  endfor -%}
{% endfor %}

Rendering gives us the same result:

# Configuring Prefix List
ip prefix-list PL_AS_65003_IN
 permit 10.96.0.0/24
 permit 10.97.11.0/24
 permit 10.99.15.0/24
 permit 10.100.5.0/25
 permit 10.100.6.128/25

And here you go, no more hardcoded references to the prefix list names! If you need another prefix list you just need to add it to the prefix_lists dictionary and it will be picked up automatically by our for loop.

Note: If you're using version of Python < 3.6 then dictionaries are not ordered. That means order in which you recorded your data might differ from the order in which items will be processed inside of a template.

If you rely on the order in which they've been recorded you should either use collections.OrderedDict if using Jinja2 in Python script, or you can apply dictsort filter in your template to order your dictionary by key or value.

To sort by key:

{% for k, v in my_dict | dictsort -%}

To sort by value:

{% for k, v in my_dict | dictsort(by='value') -%}

This concludes basics of looping in Jinja2 templates. The above use cases should cover 95% of your needs.

If you're looking for discussion of some advanced features connected to looping, rest assured I will be doing write up on those as well. I decided to leave more in depth Jinja2 topics for the final chapters of this tutorial and focus on the core stuff that lets you become productive quicker.

Conditionals and tests

Now that we're done with loops it's time to move on to conditionals.

Jinja2 implements one type of conditional statement, the if statement. For branching out we can use elif and else.

Conditionals in Jinja2 can be used in a few different ways. We'll now have a look at some use cases and how they combine with other language features.

Comparisons

First thing we look at is comparing values with conditionals, these make use of ==, !=, >, >=, <, <= operators. These are pretty standard but I will show some examples nonetheless.

One common scenario where comparison is used is varying command syntax based on the version, or vendor, of the installed OS. For instance some time ago Arista had to change a number of commands due to the lawsuit and we could use a simple if statement to make sure our templates work with all of the EOS versions:

Template, vars, and rendered template for host using EOS 4.19:

(venv) przemek@quasar:~/nauto/jinja/python$ python j2_render.py \
-t templates/eos-ver.j2 -f vars/eos-ver-419.yml -d yaml
###############################################################################
# Loaded template: templates/eos-ver.j2
###############################################################################

hostname {{ hostname }}
{% if eos_ver >= 4.22 -%}
Detected EOS ver {{ eos_ver }}, using new command syntax.
{% else -%}
Detected EOS ver {{ eos_ver }}, using old command syntax.
{% endif %}

###############################################################################
# Render variables
###############################################################################

eos_ver: 4.19
hostname: arista_old_eos

###############################################################################
# Rendered template
###############################################################################

hostname arista_old_eos
Detected EOS ver 4.19, using old command syntax.

And same for device running EOS 4.22:

(venv) przemek@quasar:~/nauto/jinja/python$ python j2_render.py \
-t templates/eos-ver.j2 -f vars/eos-ver-422.yml -d yaml
###############################################################################
# Loaded template: templates/eos-ver.j2
###############################################################################

hostname {{ hostname }}
{% if eos_ver >= 4.22 -%}
Detected EOS ver {{ eos_ver }}, using new command syntax.
{% else -%}
Detected EOS ver {{ eos_ver }}, using old command syntax.
{% endif %}

###############################################################################
# Render variables
###############################################################################

eos_ver: 4.22
hostname: arista_new_eos

###############################################################################
# Rendered template
###############################################################################

hostname arista_new_eos
Detected EOS ver 4.22, using new command syntax.

Quite simple really yet very useful. All we did is check if recorded EOS version is less than, or greater/equal than 4.22, and this is enough to make sure correct syntax makes it to the configs.

To show more complex branching with comparisons I've got here na example of template supporting multiple routing protocols where only relevant config is generated for each device.

First we define some data for hosts.

Device running BGP:

hostname: router-w-bgp
routing_protocol: bgp

interfaces:
  Loopback0: 
    ip: 10.0.0.1
    mask: 32

bgp:
  as: 65001

Device running OSPF:

hostname: router-w-ospf
routing_protocol: ospf

interfaces:
  Loopback0:
    ip: 10.0.0.2
    mask: 32

ospf:
  pid: 1

Device with default route only:

hostname: router-w-defgw

interfaces:
  Ethernet1:
    ip: 10.10.0.10
    mask: 24

default_nh: 10.10.0.1

Then we create a template using conditionals with branching. Additional protocols choices can be easily added as needed.

hostname {{ hostname }}
ip routing

{% for intf, idata in interfaces.items() -%}
interface {{ intf }}
  ip address {{ idata.ip }}/{{ idata.mask }}
{%- endfor %}

{% if routing_protocol == 'bgp' -%}
router bgp {{ bgp.as }}
  router-id {{ interfaces.Loopback0.ip }}
  network {{ interfaces.Loopback0.ip }}/{{ interfaces.Loopback0.mask }}
{%- elif routing_protocol == 'ospf' -%}
router ospf {{ ospf.pid }}
  router-id {{ interfaces.Loopback0.ip }}
  network {{ interfaces.Loopback0.ip }}/{{ interfaces.Loopback0.mask }} area 0
{%- else -%}
  ip route 0.0.0.0/0 {{ default_nh }}
{%- endif %}

Rendering results for all devices:

hostname router-w-bgp
ip routing

interface Loopback0
  ip address 10.0.0.1/32

router bgp 65001
  router-id 10.0.0.1
  network 10.0.0.1/32

hostname router-w-ospf
ip routing

interface Loopback0
  ip address 10.0.0.2/32

router ospf 1
  router-id 10.0.0.2
  network 10.0.0.2/32 area 0

hostname router-w-defgw
ip routing

interface Ethernet1
  ip address 10.10.0.10/24

ip route 0.0.0.0/0 10.10.0.1

So there you have it, one template supporting 3 different configuration options, pretty cool.

Logical operators

No implementation of conditionals would be complete without logical operators. Jinja2 provides these in the form of and, or and not.

There is not an awful lot to talk about here so here's just a short example showing all of these in action:

(venv) przemek@quasar:~/nauto/jinja/python$ python j2_render.py \
-t templates/if-logic-ops.j2 -f vars/if-logic-ops.yml
###############################################################################
# Loaded template: templates/if-logic-ops.j2
###############################################################################

{% if x and y -%}
Both x and y are True. x: {{ x }}, y: {{ y }}
{%- endif %}

{% if x or z -%}
At least one of x and z is True. x: {{ x }}, z: {{ z }}
{%- endif %}

{% if not z -%}
We see that z is not True. z: {{ z }}
{%- endif %}

###############################################################################
# Render variables
###############################################################################

x: true
y: true
z: false

###############################################################################
# Rendered template
###############################################################################

Both x and y are True. x: True, y: True

At least one of x and z is True. x: True, z: False

We see that z is not True. z: False

Truthiness

This is is a good place to look at different variable types and their truthiness. As is the case in Python, strings, lists, dictionaries, etc., variables evaluate to True if they're not empty. For empty values evaluation results in False.

I created an example illustrating thruthiness of, non-empty and empty, string, list and dictionary:

(venv) przemek@quasar:~/nauto/jinja/python$ python j2_render.py \
-t templates/if-types-truth.j2 -f vars/if-types-truth.yml
###############################################################################
# Loaded template: templates/if-types-truth.j2
###############################################################################

{% macro bool_eval(value) -%}
{% if value -%}
True
{%- else -%}
False
{%- endif %}
{%- endmacro -%}

My one element list has bool value of: {{ bool_eval(my_list) }}
My one key dict has bool value of: {{ bool_eval(my_dict) }}
My short string has bool value of: {{ bool_eval(my_string) }}

My empty list has bool value of: {{ bool_eval(my_list_empty) }}
My empty dict has bool value of: {{ bool_eval(my_dict_empty) }}
My empty string has bool value of: {{ bool_eval(my_string_empty) }}

###############################################################################
# Render variables
###############################################################################

{
    "my_list": [
        "list-element"
    ],
    "my_dict": {
        "my_key": "my_value"
    },
    "my_string": "example string",
    "my_list_empty": [],
    "my_dict_empty": {},
    "my_string_empty": ""
}

###############################################################################
# Rendered template
###############################################################################

My one element list has bool value of: True
My one key dict has bool value of: True
My short string has bool value of: True

My empty list has bool value of: False
My empty dict has bool value of: False
My empty string has bool value of: False

Personally I would advise against testing non-boolean types for truthiness. There aren't that many cases where this could be useful and it might make your intent non-obvious. If you simply want to check if the variable exists then is defined test, which we'll look at shortly, is usually a better choice.

Tests

Tests in Jinja2 are used with variables and return True or False, depending on whether the value passes the test or not. To use this feature add is and test name after the variable.

The most useful test is defined which I already mentioned. This test simply checks if given variable is defined, that is if rendering engine can find it in the data it received.

Checking if variable is defined is something I use in most of my templates. Remember that by default undefined variables will simply evaluate to an empty string. By checking if variable is defined before its intended use you make sure that your template fails during rendering. Without this test you could end with incomplete document and no indication that something is amiss.

Another family of tests that I find handy are used for checking type of the variable. Certain operations require both operands to be of the same type, if they're not Jinja2 will throw an error. This applies to things like comparing numbers or iterating over lists and dictionaries.

boolean - check is variable is a boolean
integer - check if variable is an integer
float - check if variable is a float
number - check if variable is number, will return True for both integer and float
string - check if variable is a string
mapping - check if variable is a mapping, i.e. dictionary
iterable - check if variable can be iterated over, will match string, list, dict, etc.
sequence - check if variable is a sequence

Below is an example of some variables having these tests applied:

(venv) przemek@quasar:~/nauto/jinja/python$ python j2_render.py \
-t templates/tests-type.j2 -f vars/tests-type.yml
###############################################################################
# Loaded template: templates/tests-type.j2
###############################################################################

{{ hostname }} is an iterable: {{ hostname is iterable }}
{{ hostname }} is a sequence: {{ hostname is sequence }}
{{ hostname }} is a string: {{ hostname is string }}

{{ eos_ver }} is a number: {{ eos_ver is number }}
{{ eos_ver }} is an integer: {{ eos_ver is integer }}
{{ eos_ver }} is a float: {{ eos_ver is float }}

{{ bgp_as }} is a number: {{ bgp_as is number }}
{{ bgp_as }} is an integer: {{ bgp_as is integer }}
{{ bgp_as }} is a float: {{ bgp_as is float }}

{{ interfaces }} is an iterable: {{ interfaces is iterable }}
{{ interfaces }} is a sequence: {{ interfaces is sequence }}
{{ interfaces }} is a mapping: {{ interfaces is mapping }}

{{ dns_servers }} is an iterable: {{ dns_servers is iterable }}
{{ dns_servers }} is a sequence: {{ dns_servers is sequence }}
{{ dns_servers }} is a mapping: {{ dns_servers is mapping }}

###############################################################################
# Render variables
###############################################################################

{
    "hostname": "sw-office-lon-01",
    "eos_ver": 4.22,
    "bgp_as": 65001,
    "interfaces": {
        "Ethernet1": "Uplink to core"
    },
    "dns_servers": [
        "1.1.1.1",
        "8.8.4.4",
        "8.8.8.8"
    ]
}

###############################################################################
# Rendered template
###############################################################################

sw-office-lon-01 is an iterable: True
sw-office-lon-01 is a sequence: True
sw-office-lon-01 is a string: True

4.22 is a number: True
4.22 is an integer: False
4.22 is a float: True

65001 is a number: True
65001 is an integer: True
65001 is a float: False

{'Ethernet1': 'Uplink to core'} is an iterable: True
{'Ethernet1': 'Uplink to core'} is a sequence: True
{'Ethernet1': 'Uplink to core'} is a mapping: True

['1.1.1.1', '8.8.4.4', '8.8.8.8'] is an iterable: True
['1.1.1.1', '8.8.4.4', '8.8.8.8'] is a sequence: True
['1.1.1.1', '8.8.4.4', '8.8.8.8'] is a mapping: False

You might've noticed that some of these tests might seem a bit ambiguous. For instance to test if variable is a list it is not enough to check if it's a sequence or an iterable. Strings also are both sequences and iterables. So are the dictionaries, even though vanilla Python classes them as Iterable and Mapping but not Sequence:

>>> from collections.abc import Iterable, Sequence, Mapping
>>>
>>> interfaces = {"Ethernet1": "Uplink to core"}
>>>
>>> isinstance(interfaces, Iterable)
True
>>> isinstance(interfaces, Sequence)
False
>>> isinstance(interfaces, Mapping)
True

So what all of this means? Well, I suggest the following tests for each type of variable:

  • Number, Float, Integer - these work just as expected, so choose whatever fits your use case.

  • Strings - it's enough to use string test:

{{ my_string is string }}

  • Dictionary - using mapping test is sufficient:

{{ my_dict is mapping }}

  • Lists - this is a tough one, full check should tests if variable is a sequence but at the same time it cannot be a mapping or a string:

{{ my_list is sequence and my list is not mapping and my list is not string }}

In some cases we know dictionary, or a string, is unlikely to appear so we can shorten the check by getting rid of mapping or string test:

{{ my_list is sequence and my list is not string }}
{{ my_list is sequence and my list is not mapping }}

For the full list of available tests follow the link in References.

Loop filtering

Last thing I wanted to touch on briefly are loop filtering and in operator.

Loop filtering does exactly what its name implies. It allows you to use if statement with for loop to skip elements that you're not interested in.

We could for instance loop over dictionary containing interfaces and process only the ones that have IP addresses:

(venv) przemek@quasar:~/nauto/jinja/python$ python j2_render.py \
-t templates/loop-filter.j2 -f vars/loop-filter.yml
###############################################################################
# Loaded template: templates/loop-filter.j2
###############################################################################

=== Interfaces with assigned IPv4 addresses ===

{% for intf, idata in interfaces.items() if idata.ipv4_address is defined -%}
{{ intf }} - {{ idata.description }}: {{ idata.ipv4_address }}
{% endfor %}

###############################################################################
# Render variables
###############################################################################

{
    "interfaces": {
        "Loopback0": {
            "description": "Management plane traffic",
            "ipv4_address": "10.255.255.34/32"
        },
        "Management1": {
            "description": "Management interface",
            "ipv4_address": "10.10.0.5/24"
        },
        "Ethernet1": {
            "description": "Span port - SPAN1"
        },
        "Ethernet2": {
            "description": "PortChannel50 - port 1"
        },
        "Ethernet51": {
            "description": "leaf01-eth51",
            "ipv4_address": "10.50.0.0/31"
        },
        "Ethernet52": {
            "description": "leaf02-eth51",
            "ipv4_address": "10.50.0.2/31"
        }
    }
}

###############################################################################
# Rendered template
###############################################################################

=== Interfaces with assigned IPv4 addresses ===

Loopback0 - Management plane traffic: 10.255.255.34/32
Management1 - Management interface: 10.10.0.5/24
Ethernet51 - leaf01-eth51: 10.50.0.0/31
Ethernet52 - leaf02-eth51: 10.50.0.2/31

As you can see we have 6 interfaces in total but only 4 of them have IP addresses assigned. With is defined test added to the loop we filter out interfaces with no IP addresses.

Loop filtering can be especially powerful when iterating over large payload returned from the device. In some cases you can ignore most of the elements and focus on things that are of interest.

in operator

in operator which is placed between two values can be used to check if value on the left is contained in the value on the right one. You can use it to test if an element appears in the list or if a key exists in a dictionary.

The obvious use cases for in operator is to check if something we're interested in just exists in a collection, we don't necessarily need to retrieve the item.

Looking at the previous example, we could check if Loopback0 is in the list interfaces, and if it does, we will use it to source Management Plane packets, if not we'll use Management1 interface.

Template:

{% if 'Loopback0' in interfaces -%}
sflow source-interface Loopback0
snmp-server source-interface Loopback0
ip radius source-interface Loopback0
{%- else %}
sflow source-interface Management1
snmp-server source-interface Management1
ip radius source-interface Management1
{% endif %}

Rendering results:

sflow source-interface Loopback0
snmp-server source-interface Loopback0
ip radius source-interface Loopback0

Notice that even though interfaces is a dictionary containing a lot of data we didn't iterate over it or retrieve any of the keys. All we wanted to know was the presence of Loopback0 key.

To be completely honest, the above template could use some tweaking, we essentially duplicated 3 lines of config and hardcoded interface names. That's not a very good practice, and I'll show you in the next post how we can make improvements here.

And with that we've come to the end of part 2 of the Jinja2 tutorial. Next I'll cover whitespaces, so you can make your documents look just right, and we'll continue looking at the language features. I hope you learned something useful here and do come back for more!

References