Developing NetBox Plugin - Part 3 - Adding search panel

Welcome to part 3 of my tutorial walking you through process of developing NetBox plugin. In part 2 we added basic web UI views to our BgpPeering plugin. In this post we'll add search panel to list view to allow us to search/filter Bgp Peering objects.

Developing NetBox Plugin tutorial series

• Developing NetBox Plugin - Part 1 - Setup and initial build
• Developing NetBox Plugin - Part 2 - Adding web UI pages
• Developing NetBox Plugin - Part 3 - Adding search panel
• Developing NetBox Plugin - Part 4 - Small improvements
• Developing NetBox Plugin - Part 5 - Permissions and API

Contents

• Introduction
• Filter class
• Search form class
• Adding form to list view template
  • Icon classes
• Adding filtering to list view class
• Conclusion
• Resources
• BGP Peering Plugin GitHub repository

Introduction

List view we created for displaying all Bgp Peering objects in one place is very useful. However it will become difficult to find items of interest once we have more than 30-50 objects. For that purpose we should add means of filtering objects to the ones that meet certain criteria.

Other objects in NetBox already have filtering functionality and use search panel located to the right of object tables. We'll try and replicate the NetBox look and feel when upgrading our view.

In our case, I want to be able to filter Bgp Peering objects by:

• site - This should be a drop-down list with NetBox defined sites.
• device - Again, drop-down list but with NetBox defined devices.
• local_as - BGP ASN used locally, it has to be an exact match.
• remote_as - BGP ASN used by 3rd party, it has to be an exact match.
• peer_name - Name of the peer, we want to allow partial matches.
• q - Generic query field that looks at peer name and descriptions. We want to allow partial matches here.

To make all this work we need a few components:

• filter class
• search form class
• updated list view template
• updated list view class

Filter class

First component we're going to work on is filter class. To help us with this we're using Django app called django_filters^[1]. This app makes it easier to build model based filtering that will serve as an abstraction used by our list view.

Filtering class is going to be recorded in filters.py file.

# filters.py
import django_filters
from django.db.models import Q

from dcim.models import Device, Site

from .models import BgpPeering


class BgpPeeringFilter(django_filters.FilterSet):
    """Filter capabilities for BgpPeering instances."""

    q = django_filters.CharFilter(
        method="search",
        label="Search",
    )

    site = django_filters.ModelMultipleChoiceFilter(
        field_name="site__slug",
        queryset=Site.objects.all(),
        to_field_name="slug",
    )

    device = django_filters.ModelMultipleChoiceFilter(
        field_name="device__name",
        queryset=Device.objects.all(),
        to_field_name="name",
    )

    peer_name = django_filters.CharFilter(
        lookup_expr="icontains",
    )

    class Meta:
        model = BgpPeering

        fields = [
            "local_as",
            "remote_as",
            "peer_name",
        ]

    def search(self, queryset, name, value):
        """Perform the filtered search."""
        if not value.strip():
            return queryset
        qs_filter = Q(peer_name__icontains=value) | Q(description__icontains=value)
        return queryset.filter(qs_filter)

Since all this is new, I'm going to go through the code in detail.

We start building filter by creating BgpPeeringFilter class that inherits from django_filters.FilterSet.

Next, in class Meta, we define model this filter is based on.

In fields we specify list of fields for which filters will be auto-generated. This is based on the model definitions. Any fields that don't need special treatment can go in here.

class Meta:
    model = BgpPeering

    fields = [
        "local_as",
        "remote_as",
        "peer_name",
    ]

Fields that need customization, like linking to other models, or partial matching, we will define as class attributes.

Let's have a look at those fields now.

• site - We want this to be a drop down-menu with multiple choices. To do that we make this an instance of django_filters.ModelMultipleChoiceFilter class.

  site = django_filters.ModelMultipleChoiceFilter
  

  We initialize the object with three arguments field_name, queryset and to_field_name.

  • field_name - Specifies attribute on the model field that we will filter BgpPeering objects on. Here we use site field from our model and its attribute slug. Attribute follows __ (double underscore) after name of the field:

    field_name="site__slug"
    

  • queryset - This defines collection of Site objects filter will present as filtering choices. We imported Site model from dcim.models and return all of the available objects.

  • to_field_name - Here we specify name of the attribute, slug, that filter will take from Site object and apply to field_name we specified earlier.

• device - Similar to site, it's a drop-down menu with multiple choices. Here we link to Device NetBox model from which filter will take name attribute. We filter BgpPeering objects using field/attribute combination of device__name.

• peer_name - Peer name is a character field, so we use django_filters.CharFilter class to define it.
  We want to allow case insensitive partial matches here. To do that we pass argument lookup_expr="icontains" when creating object.
  Default lookup method is exact which forces exact matches. See docs for available lookup methods ^[2].

• q - This is general, string, query field. We want it to be of character type django_filters.CharFilter.
  In method argument we specify Python method, here search, that should be called when this field is used ^[3].
  And label field is the text that will appear in the field as a hint.

  q = django_filters.CharFilter(
      method="search",
      label="Search",
  )
  

  Next we define method search, inside of the class, which q field filter will use. You can define these methods outside of the class scope but in most cases it makes sense to keep it close to the field definition.

  def search(self, queryset, name, value):
      """Perform the filtered search."""
      if not value.strip():
          return queryset
      qs_filter = Q(peer_name__icontains=value) | Q(description__icontains=value)
      return queryset.filter(qs_filter)  
  

  Methods we define for filter fields will be passed queryset, name and value arguments. Hence why our method has (self, queryset, name, value) signature.

  • queryset - This is essentially list of objects currently meeting filter criteria, before q filter is applied.
  • name - Name of the filter field, here q.
  • value - Value entered into the filter field in web GUI.

  Logic in our method is relatively simple. We return unchanged queryset if no value was provided:

  if not value.strip():
          return queryset
  

  If there is a value we take advantage of Django Q^[4] object to build a query based on two fields.

  • Q(peer_name__icontains=value) - We check, ignoring case, if peer_name field contains value.
  • Q(description__icontains=value) - We check, ignoring case, if description field contains value.

  Then we combine these objects with | - logical OR - operator and assign the result to qs_filter variable.

  qs_filter = Q(peer_name__icontains=value) | Q(description__icontains=value)

  Finally, we apply this filter to queryset and return the result.

  return queryset.filter(qs_filter)

  End effect is that if value is contained in either peer_name or peering description for given object then the object will be included in final queryset.

With that we finish filter class and move on to the form.

Search form class

Next step in our quest for filtering is creating class that will represent the search form.

We will add the below to forms.py:

# forms.py
from dcim.models import Device, Site
 
class BgpPeeringFilterForm(BootstrapMixin, forms.ModelForm):
    """Form for filtering BgpPeering instances."""

    q = forms.CharField(required=False, label="Search")

    site = forms.ModelChoiceField(
        queryset=Site.objects.all(), required=False, to_field_name="slug"
    )

    device = forms.ModelChoiceField(
        queryset=Device.objects.all(),
        to_field_name="name",
        required=False,
    )

    local_as = forms.IntegerField(
        required=False,
    )

    remote_as = forms.IntegerField(
        required=False,
    )

    peer_name = forms.CharField(
        required=False,
    )

    class Meta:
        model = BgpPeering
        fields = []

Let's break this code down.

We create BgpPeeringFilterForm that inherits from BootstrapMixin and forms.ModelForm. This is just like the form we defined in part 2 of this tutorial.

Next we define model we're building this form from, BgpPeering, and fields that will be auto-generated. I want to define all fields explicitly so fields attribute will be an empty list. You need to include this attribute even if you don't list anything here or an exception will be raised.

class Meta:
    model = BgpPeering
    fields = []

Following that we will define types and attributes of the fields used by search form.

• q - General query field should be a character field so we use type forms.CharField. This field is optional and we manually set label of this field to Search.

• site - Site selection field, this field is optional.

  • To link it to NetBox Site objects we make it of type forms.ModelChoiceField.
  • We ask for all Site objects to be available in drop-down with queryset=Site.objects.all().
  • When given item is selected we want slug attribute to be returned. This is what to_field_name="slug" does.

• device - Similar to site but here we link to Device model and ask for name attribute to be returned.

• local_as and remote_as - Optional integer fields, we use type forms.IntegerField for those.

• peer_name - Simple, optional, character field. We use type forms.CharField for it.

And that's our form class done.

Adding form to list view template

To allow users to search through BgpPeering objects we need to update our web UI page. We'll add search panel on the right side of the object list view template.

We add new <div> in bgppeering_list.html right after the one we created in part 2:

<div class="col-md-9">
    {% render_table table %}
</div>
<!-- search panel div start -->
<div class="col-md-3 noprint">
    <div class="panel panel-default">
        <div class="panel-heading">
            <span class="{{ icon_classes.search }}" aria-hidden="true"></span>
            <strong>Search</strong>
        </div>
        <div class="panel-body">
            <form action="." method="get" class="form">
                {% for field in filter_form.visible_fields %}
                <div class="form-group">
                    {% if field.name == "q" %}
                    <div class="input-group">
                        <input type="text" name="q" class="form-control" placeholder="{{ field.label }}"
                            {% if request.GET.q %}value="{{ request.GET.q }}" {% endif %} />
                        <span class="input-group-btn">
                            <button type="submit" class="btn btn-primary">
                                <span class="{{ icon_classes.search }}" aria-hidden="true"></span>
                            </button>
                        </span>
                    </div>
                    {% else %}
                    {{ field.label_tag }}
                    {{ field }}
                    {% endif %}
                </div>
                {% endfor %}
                <div class="text-right noprint">
                    <button type="submit" class="btn btn-primary">
                        <span class="{{ icon_classes.search }}" aria-hidden="true"></span> Apply
                    </button>
                    <a href="." class="btn btn-default">
                        <span class="{{ icon_classes.remove }}" aria-hidden="true"></span> Clear
                    </a>
                </div>
            </form>
        </div>
    </div>
</div>

We make this div 3 columns wide. Inside we place panel header and search form divs.

Form fields are rendered in for loop:

{% for field in filter_form.visible_fields %}
...
{% endfor %}

These fields are taken from form class we defined earlier. We leave all fields, with exception of q field, to their defaults by displaying field label followed by rendering actual field.

{{ field.label_tag }}
{{ field }}

Because q field does not belong to underlying model we handle it differently. We make it a text input field with label as a placeholder.

The interesting bit here is that the value of the field is carried over from previous search. This is done for us for auto-generated form fields but here we have to do it manually with the below expression:

{% if request.GET.q %}value="{{ request.GET.q }}" {% endif %} />

Remaining html/css code is for layout and visual elements.

Icon classes

You might have noticed a few references to icon_classes variable here like in class="{{ icon_classes.search }}". These are strings specifying CSS classes used for rendering icons. I pass these classes to the form in icon_classes variable from form view.

NetBox v2.10+ uses Material Design icons. Previous versions used Font Awesome. To make my plugin compatible with both versions I created icon_classes.py file with dictionary dynamically mapping class names to underlying MD or FA CSS classes.

# icon_classes.py
from .release import NETBOX_RELEASE_CURRENT, NETBOX_RELEASE_210


if NETBOX_RELEASE_CURRENT >= NETBOX_RELEASE_210:
    icon_classes = {
        "plus": "mdi mdi-plus-thick",
        "search": "mdi mdi-magnify",
        "remove": "mdi mdi-close-thick",
    }
else:
    icon_classes = {
        "plus": "fa fa-plus",
        "search": "fa fa-search",
        "remove": "fa fa-remove",
    }

I also use release.py borrowed from ^[5] to detect version of NetBox plugin is running under.

from packaging import version
from django.conf import settings

NETBOX_RELEASE_CURRENT = version.parse(settings.VERSION)
NETBOX_RELEASE_28 = version.parse("2.8")
NETBOX_RELEASE_29 = version.parse("2.9")
NETBOX_RELEASE_210 = version.parse("2.10")

The above additions allow me to easily add CSS classes for any other icons I might want to use in the future.

With that digression out of the way, let's put it all together by modifying list view class.

Adding filtering to list view class

All of the components we created up to this point need to be tied together in the class view. We're modifying the BgpPeeringListView class created in part 2, in order to support search/filtering functionality.

Changed BgpPeeringListView class:

# views.py
from .icon_classes import icon_classes
from .filters import BgpPeeringFilter
from .forms import BgpPeeringForm, BgpPeeringFilterForm

class BgpPeeringListView(View):
    """View for listing all existing BGP Peerings."""

    queryset = BgpPeering.objects.all()
    filterset = BgpPeeringFilter
    filterset_form = BgpPeeringFilterForm

    def get(self, request):
        """Get request."""

        self.queryset = self.filterset(request.GET, self.queryset).qs

        table = BgpPeeringTable(self.queryset)
        RequestConfig(request, paginate={"per_page": 25}).configure(table)

        return render(
            request,
            "netbox_bgppeering/bgppeering_list.html",
            {
                "table": table,
                "filter_form": self.filterset_form(request.GET),
                "icon_classes": icon_classes,
            },
        )

Few interesting things happen here, let's break them down.

• filterset = BgpPeeringFilter - We add filterset attribute and set it to BgpPeeringFilter class we created earlier.

• filterset_form = BgpPeeringFilterForm - This is the form that will be rendered in list view template.

• self.queryset = self.filterset(request.GET, self.queryset).qs - Here is where the filtering happens.
  We feed filterset form values contained in request.GET and queryset with BgpPeering objects.
  Method .qs returns QuerySet like object that we assign back to self.queryset. This will be then fed to table constructor. Except now the resulting table will only contain objects matching filter values.

Finally, we provide two new arguments to render:

• "filter_form": self.filterset_form(request.GET) - This is used to render form in UI. request.GET preserves values used in previous searches.

• "icon_classes": icon_classes - This passes dictionary with CSS classes I defined for UI icons.

And that's it. We can now re-build plugin and take search panel for a spin.

If you now navigate to /plugins/bgp-peering/ you should see search panel on the right hand side, next to table with the list of objects.

And here's the table after we asked for peerings on one device only.

And here's a result of the general query for primary string.

All working as intended, pretty cool right?

Source code with all the modifications we made up to this point is in branch bgppeering-list-search if you want to check it out: https://github.com/progala/ttl255-netbox-plugin-bgppeering/tree/bgppeering-list-search

Conclusion

This concludes post 3 in Developing NetBox Plugin series. Search functionality we implemented here should greatly improve usability of our plugin. It's going to be much easier now to find peering objects of interest.

In the next post we'll continue with making improvements to the plugin. There a few small things here and there that will make this all even better. Stay tuned!

Resources

• NetBox docs: Plugin Development: https://netbox.readthedocs.io/en/stable/plugins/development/
• NetBox source code on GitHub: https://github.com/netbox-community/netbox
• NetBox Extensibility Overview, NetBox Day 2020: https://www.youtube.com/watch?v=FSoCzuWOAE0
• NetBox Plugins Development, NetBox Day 2020: https://www.youtube.com/watch?v=LUCUBPrTtJ4

---

1. Django Filter app - https://django-filter.readthedocs.io/en/stable/ ↩︎

2. Django field lookups - https://docs.djangoproject.com/en/3.1/ref/models/querysets/#field-lookups ↩︎

3. Django filter method - https://django-filter.readthedocs.io/en/stable/ref/filters.html#method ↩︎

4. Django Q object - https://docs.djangoproject.com/en/3.1/topics/db/queries/#complex-lookups-with-q ↩︎

5. NTC NetBox Onboarding plugin - https://github.com/networktocode/ntc-netbox-plugin-onboarding/blob/develop/netbox_onboarding/release.py ↩︎