Network Automation: Network APIs and Scripting

Introduction: The Shift from Manual to Programmatic Networking

For decades, network engineering was defined by the Command Line Interface (CLI). Engineers would log into individual switches, routers, and firewalls, typing commands one by one to configure interfaces, update routing tables, or troubleshoot connectivity. While this approach served the industry well during the era of static, perimeter-based networks, it has become a bottleneck in the modern data center and cloud-centric world. Today’s networks are dynamic, requiring frequent changes to support virtualization, containerization, and rapidly scaling application environments.

Network automation is the process of using software to plan, configure, and manage network infrastructure. At the heart of this transition are Network APIs and Scripting. By moving away from human-driven CLI interactions toward machine-readable interfaces, engineers can treat network infrastructure like code. This allows for version control, automated testing, and the ability to deploy complex configurations across hundreds of devices in seconds rather than hours. This lesson explores the fundamental building blocks of this transition: how we interact with network devices programmatically and the tools we use to glue these interactions together.

Understanding these concepts is not just about staying relevant in the job market; it is about reducing human error. Studies consistently show that the vast majority of network outages are caused by manual configuration errors. When you replace a manual "copy-paste" workflow with a scripted, validated, and automated process, you remove the variability of human input, leading to a much more stable and predictable network environment.

The Evolution of Network Programmability

To understand why APIs and scripting are the future, we must first look at the mechanisms we are replacing and augmenting. Traditional management protocols like SNMP (Simple Network Management Protocol) were designed for monitoring, not configuration. While SNMP can set values, it is notoriously difficult to use for complex device orchestration.

The CLI Scraping Problem

Most engineers started their automation journey with "screen scraping." This involves using a script (often in Python using libraries like Netmiko or Paramiko) to log into a device via SSH, send CLI commands, read the output string, and parse it using Regular Expressions (Regex). While this works, it is fragile. If a vendor updates the firmware and the CLI output format changes slightly, your entire automation pipeline breaks.

The Rise of APIs

Application Programming Interfaces (APIs) solve the fragility of screen scraping by providing a structured, predictable way to interact with network devices. Instead of sending human-readable text and trying to interpret the response, APIs allow you to send structured data—usually in JSON or XML format—and receive structured data back. This is the cornerstone of the "Programmable Network."

Callout: CLI vs. API Interactions

When using a CLI, you are acting as a human operator, typing commands and interpreting visual text. When using an API, you are acting as a software developer, sending structured requests to an endpoint. CLI is stateful and interactive, whereas APIs are typically stateless and transaction-based, making them significantly easier to automate and test within a continuous integration pipeline.

Core Technologies: REST APIs and Data Models

If you are going to automate a network, you must speak the language of the devices. Modern network devices expose their configuration through APIs, most commonly REST (Representational State Transfer) APIs.

Understanding REST APIs

A REST API uses standard HTTP methods to perform operations on resources. Think of a network device as a collection of resources, such as interfaces, VLANs, or routing protocols. You use HTTP verbs to tell the device what to do with these resources:

• GET: Retrieve information (e.g., "Show me the status of interface GigabitEthernet0/1").
• POST: Create a new resource (e.g., "Create a new VLAN 100").
• PUT: Replace an existing resource (e.g., "Overwrite the entire configuration of interface 0/1").
• PATCH: Update a specific part of a resource (e.g., "Change the description of interface 0/1").
• DELETE: Remove a resource (e.g., "Remove VLAN 100").

Data Formats: JSON and XML

APIs don't send raw text; they send structured data. JSON (JavaScript Object Notation) has become the industry standard because it is lightweight, easy for humans to read, and maps directly to dictionary/list structures in languages like Python.

{
  "interface": {
    "name": "GigabitEthernet0/1",
    "description": "Uplink_to_Core",
    "enabled": true,
    "mtu": 1500
  }
}

By sending this JSON payload to an API endpoint, you are telling the network device exactly how the interface should be configured. The device handles the translation of this data into its internal operating system, eliminating the need for you to worry about specific CLI syntax.

Scripting for Network Automation: The Python Advantage

While there are many tools available, Python has emerged as the dominant language for network automation. Its simple syntax, massive ecosystem of libraries, and strong support for network protocols make it the logical choice for both beginners and experts.

Essential Python Libraries

To get started, you don't need to write everything from scratch. The Python community has developed robust libraries that handle the heavy lifting of network communication.

• Requests: The standard library for making HTTP requests. You will use this to talk to REST APIs.
• Netmiko: A multi-vendor library that simplifies SSH connections to legacy devices. It handles the quirks of different vendor CLIs so you don't have to.
• NAPALM: An abstraction layer that provides a unified API for interacting with different network operating systems (Cisco IOS, Juniper Junos, Arista EOS, etc.).
• Nornir: A framework designed for concurrency, allowing you to run tasks across thousands of devices simultaneously.

Practical Example: Using Python to Query an API

Let's look at a simple Python script using the requests library to get information from a device with a REST API.

import requests
import json

# Define the device details
url = "https://192.168.1.1/restconf/data/interfaces/interface=GigabitEthernet1"
headers = {
    "Accept": "application/yang-data+json",
    "Content-Type": "application/yang-data+json"
}
auth = ("admin", "password123")

# Send the GET request
response = requests.get(url, headers=headers, auth=auth, verify=False)

# Check if the request was successful
if response.status_code == 200:
    data = response.json()
    print(json.dumps(data, indent=2))
else:
    print(f"Error: {response.status_code}")

In this example, we define the URL, authentication, and the headers required by the API. The script sends the request, checks the status code to ensure success, and then prints the returned JSON data in a readable format. This is the foundation of network monitoring and data collection.

Step-by-Step: Automating a Configuration Change

Now that we understand how to query a device, let's look at how to push a configuration change. This is where the power of automation truly shines.

Step 1: Define the Desired State

Before you write code, you must define what the final configuration should look like. In automation, we call this the "Desired State." Instead of writing a script that says "add this VLAN," you write a script that says "ensure VLAN 100 exists."

Step 2: Use a Configuration Template

Hardcoding configuration values into your script is a bad practice. Instead, use a templating engine like Jinja2. Jinja2 allows you to create a template file where variables are placeholders.

Template (vlan_config.j2):

vlan {{ vlan_id }}
 name {{ vlan_name }}
 state active

Python Code:

from jinja2 import Template

template_content = open('vlan_config.j2').read()
template = Template(template_content)

# Render the template with specific data
config = template.render(vlan_id=100, vlan_name="Data_VLAN")
print(config)

Step 3: Apply the Configuration

Using a tool like Netmiko or a vendor-specific API, you send the rendered configuration to the device.

from netmiko import ConnectHandler

device = {
    'device_type': 'cisco_ios',
    'host': '192.168.1.1',
    'username': 'admin',
    'password': 'password123'
}

net_connect = ConnectHandler(**device)
output = net_connect.send_config_set(config.splitlines())
print(output)

By decoupling the template from the data, you can reuse the same script for hundreds of different VLANs just by changing the input variables.

Best Practices for Network Automation

Automation is powerful, but it can also be dangerous. A script that misconfigures one interface is a nuisance; a script that misconfigures the routing table on every core switch in your data center is a catastrophe.

1. Version Control is Mandatory

Never run a script that isn't stored in a version control system like Git. Git allows you to track changes, revert to previous versions if something goes wrong, and collaborate with your team. If you are not using Git, you are not doing professional-grade automation.

2. Implement the "Test, Stage, Prod" Workflow

Never run automation directly against production devices. You should have a lab environment (or at least a virtual network simulator like GNS3, EVE-NG, or Cisco CML) where you can test your scripts. Once the script works in the lab, move it to a staging environment before finally deploying to production.

3. Use Idempotency

An idempotent operation is one that can be applied multiple times without changing the result beyond the initial application. For example, a script that checks if a VLAN exists before trying to create it is idempotent. A script that tries to create the VLAN every time it runs—and crashes if the VLAN already exists—is not. Always write your scripts to verify the current state before applying changes.

4. Start Small

Don't try to automate the entire network at once. Start with "Read-Only" automation—scripts that collect data, inventory devices, or check for health status. Once you are comfortable reading data, move on to low-risk configuration changes, such as updating descriptions or NTP settings.

Warning: The "Blast Radius" Principle

Always consider the "blast radius" of your automation. If a script fails, how many devices are impacted? Use batching to deploy changes to a small subset of the network first (e.g., 5% of devices), verify the health of the network, and then proceed to the rest.

Comparison of Automation Approaches

Common Pitfalls and How to Avoid Them

Pitfall 1: Hardcoding Credentials

Never put usernames and passwords directly into your scripts. If you accidentally upload your script to a shared repository, your network credentials are compromised. Use environment variables or secret management tools like HashiCorp Vault.

Pitfall 2: Ignoring Error Handling

What happens when a device is unreachable? What if the API returns a 500 error? If your script doesn't handle these scenarios, it will crash and leave your network in an inconsistent state. Always use try-except blocks in Python to catch errors and log them appropriately.

Pitfall 3: Lack of Logging

When an automation task fails, you need to know exactly why. Build logging into your scripts. Record the time, the device, the command sent, and the response received. This audit trail is essential for troubleshooting.

Pitfall 4: Over-Automating

Not everything should be automated. If a task takes five minutes to do manually and occurs once a year, the time required to write, test, and maintain an automation script for it is not a good return on investment. Focus your automation efforts on high-frequency, high-complexity tasks.

Deep Dive: Model-Driven Programmability (YANG and NETCONF)

While REST APIs are excellent for general-purpose automation, they are often vendor-specific. This is where YANG (Yet Another Next Generation) data modeling and NETCONF come in.

YANG is a data modeling language used to define the configuration and state data of a network device. Think of it as a schema or a blueprint for the device's configuration. Because YANG models are standardized (e.g., IETF models), you can theoretically use the same data structure to configure a Cisco switch and a Juniper router.

NETCONF is the transport protocol that carries this data. It provides a secure, reliable way to install, manipulate, and delete the configuration of network devices. Unlike REST, which is built for web applications, NETCONF is built specifically for the needs of network infrastructure, supporting features like configuration validation and atomic transactions (where the entire configuration is applied at once, or not at all).

Callout: RESTCONF vs. NETCONF

RESTCONF is essentially a bridge between the two worlds. It provides a RESTful API (using HTTP) that allows you to interact with YANG-modeled data. It is often the preferred choice for engineers who want the structure of YANG but the ease of use of a standard REST API.

Advanced Scripting: Working with Concurrency

One of the biggest advantages of automation is speed. However, if you write a standard Python script that connects to devices one by one, you lose this advantage. If it takes 5 seconds to connect to a device and you have 1,000 devices, your script will take over an hour to finish.

To solve this, we use Concurrency. Python provides the threading or asyncio libraries, but for network automation, specialized frameworks like Nornir are the industry standard. Nornir allows you to define a set of tasks and run them across all your devices in parallel.

# Example of conceptual Nornir usage
from nornir import InitNornir
from nornir_netmiko.tasks import netmiko_send_config

nr = InitNornir(config_file="config.yaml")

def deploy_config(task):
    task.run(task=netmiko_send_config, config_commands=["ntp server 10.0.0.1"])

# Run the task across all devices in the inventory
nr.run(task=deploy_config)

In this example, Nornir handles the connection pool, the threading, and the error reporting. You simply define the "what," and the framework handles the "how."

Integrating Automation into the Network Lifecycle

Automation should not be an isolated event; it should be part of the network lifecycle. This lifecycle generally follows these phases:

1. Design: Define the network topology and policies as code (using tools like NetBox for Source of Truth).
2. Provision: Use automation scripts (Python, Ansible, Terraform) to push configurations to hardware.
3. Validate: Run automated tests to ensure the configuration was applied correctly and the network is functioning as expected (using tools like Batfish or pyATS).
4. Monitor: Use telemetry (streaming data) to monitor the health of the network in real-time.
5. Remediate: If an issue is detected, trigger automated workflows to fix the problem or isolate the affected segment.

By integrating these steps, you create a "Closed-Loop" automation system where the network can essentially manage itself based on the policies you define.

Frequently Asked Questions (FAQ)

Do I need to be a software developer to learn network automation?

No. You do not need to be a software developer, but you do need to adopt a "developer mindset." This means focusing on modular code, documentation, testing, and version control. You are an engineer who uses code as a tool, not a full-time software engineer.

Is CLI dead?

Not at all. You will always need the CLI for deep-dive troubleshooting or when APIs are unavailable. Automation is about augmenting your capabilities, not replacing your fundamental networking knowledge.

Which language should I learn first?

Python is the undisputed leader in network automation. Start there. Once you are comfortable with Python, you might explore configuration management tools like Ansible, which are built on top of Python but provide a simpler, declarative way to manage large-scale infrastructure.

What is a "Source of Truth"?

A Source of Truth is a centralized database that contains the intended state of your network. Instead of querying a device to see what its IP address is, you query your Source of Truth (like NetBox). If the device doesn't match the Source of Truth, your automation script should update the device to match the record.

Practical Checklist for Getting Started

1. Set up a Virtual Lab: Download GNS3 or EVE-NG. You cannot learn automation without a safe place to break things.
2. Learn Basic Python: Focus on data structures (lists, dictionaries), loops, and error handling.
3. Install Git: Learn the basics of git init, git add, git commit, and git push.
4. Practice with Netmiko: Start by writing a script that logs into your lab devices and runs a simple show command.
5. Transition to APIs: Once comfortable with SSH-based automation, find a device with a REST API and start making GET requests.
6. Join the Community: The network automation community is very active. Participate in forums, follow blogs from network engineers, and contribute to open-source projects on GitHub.

Key Takeaways

• Automation is a Mindset: It is about moving from manual, repetitive CLI tasks to repeatable, version-controlled, and tested workflows.
• APIs are the Standard: REST APIs and structured data formats like JSON provide a reliable, programmatic way to interact with network devices, replacing the fragile nature of screen scraping.
• Python is the Tool of Choice: Its readability and the availability of specialized libraries like Netmiko and Nornir make it the essential language for modern network engineers.
• Safety First: Always use version control (Git), test in a lab environment before production, and implement idempotency to ensure your scripts don't cause unintended side effects.
• Start with Read-Only: Don't jump straight into modifying core infrastructure. Begin by automating data collection and inventory, then progress to configuration changes as your confidence and testing framework mature.
• Embrace the Lifecycle: Automation is most effective when integrated into a full lifecycle, from design and provisioning to validation and continuous monitoring.
• The Source of Truth Matters: Effective automation relies on accurate data. Centralizing your network configuration data in a tool like NetBox is a prerequisite for high-scale, reliable automation.

By following these principles, you will move from being a CLI-bound operator to an architect of programmable infrastructure. The journey takes time, but the result—a more resilient, scalable, and manageable network—is well worth the effort. Start small, stay consistent, and always prioritize the stability of the network.