Site-to-Site VPN Configuration

Lesson: Site-to-Site VPN Configuration for Hybrid Networks

Introduction: Bridging the Gap Between On-Premises and Cloud

In the modern landscape of infrastructure management, very few organizations operate entirely within a single environment. Most enterprises rely on a hybrid architecture, where sensitive data or legacy applications reside in an on-premises data center, while scalable, elastic workloads run in public cloud environments like Azure, AWS, or Google Cloud. To make these two disparate environments function as a single, cohesive network, we need a secure, reliable, and persistent communication channel. This is where the Site-to-Site (S2S) VPN comes into play.

A Site-to-Site VPN acts as a secure tunnel over the public internet, encrypting traffic between your local network gateway and the cloud-based virtual private gateway. Unlike a client-to-site VPN, which connects an individual user to a network, an S2S VPN connects entire network segments. It allows your on-premises servers to communicate with cloud-based virtual machines using private IP addresses as if they were sitting in the same rack in your local server room. Understanding how to design, configure, and maintain these tunnels is a fundamental skill for any infrastructure engineer, as it directly impacts connectivity, security, and performance.

Why Site-to-Site VPNs Matter

The primary value of an S2S VPN is transparency. When configured correctly, applications do not need to know whether they are talking to a service across the street or across the globe. By extending your corporate network into the cloud, you can manage hybrid resources with your existing tools, such as Active Directory, centralized monitoring systems, and internal DNS servers. Furthermore, because the traffic is encrypted, you can maintain compliance with security policies that require data in transit to be protected, even when crossing public network infrastructure.

However, the simplicity of the concept masks the complexity of the implementation. Factors like routing protocols, encryption standards, public IP address management, and tunnel stability require careful planning. If you ignore these nuances, you will likely encounter fragmented packets, connection drops, or security vulnerabilities that could compromise your entire infrastructure. This lesson will guide you through the technical requirements and practical steps to deploy a stable and secure S2S VPN tunnel.

The Architecture of a Site-to-Site VPN

To understand how a Site-to-Site VPN works, we must look at the two endpoints involved in the communication. On your on-premises side, you typically have a hardware appliance, such as a Cisco ASA, Juniper SRX, or a Linux-based firewall running StrongSwan. On the cloud side, you have a Virtual Private Gateway or a managed VPN Gateway service provided by your cloud vendor.

The connection process involves two distinct phases of negotiation, commonly referred to as Phase 1 and Phase 2. These phases are governed by the Internet Key Exchange (IKE) protocol.

Internet Key Exchange (IKE) Phases

• IKE Phase 1 (The Handshake): In this stage, the two gateways authenticate each other and establish a secure, authenticated channel for further communication. They agree on encryption algorithms (like AES), hashing algorithms (like SHA-256), and Diffie-Hellman groups to exchange keys securely.
• IKE Phase 2 (The Tunnel): Once the secure channel is established, Phase 2 negotiates the specific security parameters for the actual data traffic. This is where the "Security Association" (SA) is created, which defines how the data packets will be encapsulated (usually via IPsec) and what traffic is permitted to traverse the tunnel.

Callout: Policy-Based vs. Route-Based VPNs

It is crucial to distinguish between these two types of VPNs early in your design phase. Policy-Based VPNs define the traffic that should be encrypted based on access control lists (ACLs) that specify source and destination IP ranges. If the traffic matches the ACL, it is encrypted. Route-Based VPNs treat the VPN tunnel as a virtual network interface. You route traffic to this interface just as you would to a physical Ethernet port. Route-based VPNs are significantly more flexible, as they support dynamic routing protocols like BGP, making them the industry standard for modern cloud deployments.

Prerequisites for Configuration

Before you begin typing commands or filling out forms in a cloud console, you must gather specific information. Without these details, you will be unable to establish a connection.

1. Public IP Address (On-Premises): Your on-premises gateway must have a static, public-facing IP address. This address acts as the endpoint that the cloud gateway will attempt to reach.
2. Private IP Address Ranges (CIDR Blocks): You must define the specific subnets on both sides of the tunnel. For example, your on-premises network might be 10.1.0.0/16, and your cloud VPC might be 10.2.0.0/16. Ensure there is no overlap, as overlapping subnets will cause routing conflicts that are notoriously difficult to debug.
3. Shared Secret (Pre-Shared Key): This is a long, complex string of characters used to authenticate the two gateways. Treat this as a high-security password; rotate it regularly and never store it in plain text within your source control systems.
4. IKE Version: Most modern gateways support IKEv2. Always prefer IKEv2 over IKEv1, as it provides better security, faster re-keying, and improved reliability during network interruptions.

Step-by-Step Implementation: A Practical Example

While specific steps vary between cloud providers, the logic remains consistent. Let’s look at a generalized workflow for setting up a route-based S2S VPN using a cloud-managed gateway.

Step 1: Create the Virtual Network Gateway

In your cloud management portal, you will first provision a VPN Gateway resource. This resource acts as the cloud-side endpoint. You will need to associate this gateway with a specific virtual network and a public IP address.

Step 2: Define the Local Network Gateway

Next, you define the "Local Network Gateway" (or "Customer Gateway" in AWS terms). This object represents your physical on-premises network. You will input the public IP address of your local firewall and the internal IP ranges (CIDR blocks) that exist on your local network.

Step 3: Configure the Connection Object

The connection object is the bridge between the Virtual Network Gateway and the Local Network Gateway. This is where you configure the specific parameters:

• Connection Type: Site-to-Site (IPsec).
• Shared Key: The pre-shared key you generated earlier.
• IKE Policy: Select the encryption and hash algorithms (e.g., AES-256, SHA-256, Diffie-Hellman Group 14 or higher).

Step 4: Configure the On-Premises Firewall

Now, move to your local hardware. You must configure your firewall to accept the VPN connection from the cloud gateway’s public IP. You will need to match the IKE/IPsec policies exactly as you defined them in the cloud console. If the settings do not match perfectly, the tunnel will fail to come up.

Warning: The "Perfect Match" Rule

A common point of failure is a mismatch in the crypto profile. If the cloud side is set to AES-256 and the on-premises side is set to AES-128, or if the Diffie-Hellman groups are different, the tunnel will negotiate Phase 1 or 2 and then immediately drop. Always verify these settings side-by-side in your documentation before troubleshooting connectivity.

Code Example: StrongSwan Configuration (Linux)

If you are using a Linux-based firewall (like a hardened Ubuntu server) on-premises, you will likely use StrongSwan for your IPsec implementation. Below is a simplified configuration snippet for /etc/ipsec.conf.

# /etc/ipsec.conf configuration example
conn cloud-to-onprem
    authby=secret
    left=%defaultroute
    leftid=<ON_PREM_PUBLIC_IP>
    right=<CLOUD_GATEWAY_PUBLIC_IP>
    rightsubnet=10.2.0.0/16
    leftsubnet=10.1.0.0/16
    ike=aes256-sha256-modp2048!
    esp=aes256-sha256-modp2048!
    keyexchange=ikev2
    auto=start

Explanation of the code:

• authby=secret: Specifies that we are using a Pre-Shared Key (defined in ipsec.secrets).
• leftid: The public IP of your local gateway.
• rightsubnet: The network range of your cloud VPC.
• leftsubnet: The network range of your local site.
• ike and esp: These lines define the exact encryption and integrity algorithms. The ! at the end tells StrongSwan to only accept these specific settings, rejecting any others.

Best Practices for Site-to-Site VPNs

To ensure your hybrid network remains stable and performant, follow these industry-standard recommendations.

1. Implement Redundancy

A single VPN tunnel is a single point of failure. If your ISP has an outage or your hardware restarts, your cloud connectivity disappears. Always deploy two tunnels (Active-Active) to two different cloud gateway instances if possible. Configure your local firewall to perform health checks on the tunnels and failover automatically if one goes down.

2. Use BGP for Dynamic Routing

Static routes are brittle. If you add a new subnet to your cloud environment, you have to manually update your on-premises firewall routing table. By using Border Gateway Protocol (BGP), you allow the two gateways to automatically exchange routing information. If a new path becomes available or a subnet is removed, the network updates itself without human intervention.

3. Monitor Tunnel Health

Most cloud providers offer metrics for VPN tunnels, such as "Tunnel Up/Down" status, "Bytes In," and "Bytes Out." Set up alerts in your monitoring system (like Prometheus, CloudWatch, or Datadog) to notify you immediately if a tunnel goes down. A silent failure is much worse than a loud one.

4. Optimize MTU and MSS

VPN tunnels add overhead to every packet due to the encryption headers (IPsec encapsulation). This reduces the effective Maximum Transmission Unit (MTU) size. If you don't adjust your Maximum Segment Size (MSS) clamping on your firewall, you will experience "black hole" issues where small packets (like pings) pass through, but large packets (like web requests or database queries) hang indefinitely.

Tip: Adjusting MSS Clamping

A common rule of thumb is to set your MSS clamping to 1350 or 1360 bytes. This ensures that the total packet size, including the IPsec overhead, does not exceed the standard 1500-byte MTU of the internet, preventing fragmentation.

Common Pitfalls and Troubleshooting

Even with careful planning, things can go wrong. Here is how to approach the most frequent issues.

"My tunnel is up, but I can't ping my VMs."

This is almost always a routing or firewall issue.

• Check the Security Groups/Firewall Rules: Do your cloud-side security groups allow ICMP traffic from your on-premises CIDR? Do your local firewall rules permit traffic from the cloud CIDR?
• Check the Route Table: Does your cloud route table know that traffic for 10.1.0.0/16 should be sent to the VPN Gateway?

"The tunnel flaps (connects and disconnects repeatedly)."

This is often caused by DPD (Dead Peer Detection) settings. If one side thinks the other is dead and initiates a re-keying process, but the other side is still active, you can get into a race condition. Ensure that your DPD settings are synchronized across both gateways.

"I have overlapping IP addresses."

If your cloud VPC uses 10.0.0.0/16 and your on-premises network also uses 10.0.0.0/16, you have a major problem. You cannot route between them. The only real solution is to re-address one of the networks. If that is impossible, you may need to implement complex NAT (Network Address Translation) rules, which are difficult to manage and prone to errors. Avoid this at all costs during the design phase.

Comparison: VPN Gateway vs. Dedicated Connections

When your traffic volume increases, you might wonder if a VPN is still the right choice. Here is a quick reference to help you decide when to stick with a VPN and when to move to a dedicated private connection (like AWS Direct Connect or Azure ExpressRoute).

• When to use a VPN: For development environments, low-to-medium traffic workloads, or when you need to connect to the cloud quickly without waiting for physical fiber installation.
• When to use a Dedicated Connection: For high-throughput applications, consistent low-latency requirements (like real-time financial trading or large-scale data migrations), or when regulatory requirements forbid traffic from traversing the public internet.

Advanced Security Considerations

While IPsec provides strong encryption for data in transit, the security of your VPN is only as good as your key management and access policies.

Key Rotation

Do not use the same pre-shared key for years. Implement a process to rotate your keys quarterly. In a production environment, this should be done during a maintenance window to minimize the brief downtime that occurs during the re-keying process.

Restricting Traffic with ACLs

Just because a tunnel exists does not mean all traffic should be allowed. Even over a VPN, you should apply the principle of least privilege. Configure your cloud-side security groups and on-premises firewall policies to only allow the specific ports and protocols necessary for your applications. For example, if you only need database traffic, open port 3306 or 5432, not the entire range of ports.

Traffic Inspection

If your organization requires deep packet inspection (DPI) for compliance, you should route your VPN traffic through a virtual firewall appliance in the cloud. This allows you to inspect traffic flowing between the cloud and on-premises environments for malware, unauthorized access attempts, or data exfiltration.

Callout: The "VPN as a Perimeter" Myth

Never assume that traffic coming from your "trusted" on-premises network is inherently safe. If a workstation on your local network is compromised, the attacker could use the VPN tunnel to pivot into your cloud environment. Always maintain a Zero Trust mindset, applying security controls at the workload level, not just at the network perimeter.

Practical Checklist for Deployment

To wrap up the implementation process, keep this checklist handy during your next deployment:

1. Network Audit: Confirm CIDR blocks on both sides do not overlap.
2. Public IP Verification: Ensure your on-premises gateway has a stable public IP.
3. Policy Alignment: Document encryption, hashing, and DH group settings.
4. Security Rules: Configure Firewall/Security Groups to allow traffic from the remote network.
5. Routing: Update route tables on both ends to point to the VPN gateway.
6. Verification: Perform a ping test and check tunnel status logs.
7. Alerting: Configure monitoring for tunnel uptime.

Industry Standards and Compliance

When operating in regulated industries (such as healthcare, finance, or government), your VPN configuration may be subject to specific standards like HIPAA, PCI-DSS, or SOC2. These standards generally require that data in transit is protected using "strong cryptography."

• Avoid Weak Ciphers: Disable support for legacy ciphers like DES, 3DES, or MD5. These are considered broken and will fail most modern compliance audits.
• Use Modern DH Groups: Use Diffie-Hellman Group 14 (2048-bit) or higher. Groups 1, 2, and 5 are insufficient for modern security requirements.
• Logging and Auditing: Ensure that your VPN gateway logs are being shipped to a centralized logging server (like an ELK stack or a SIEM). You must be able to audit who connected and when, especially if the VPN is used by remote administrators.

Handling Connection Issues: A Troubleshooting Methodology

When a VPN fails, it is easy to start "knob-twiddling"—changing random settings in hopes that it works. This is the wrong approach. Follow a structured methodology instead:

1. Verify Physical/Logical Connectivity: Can you ping the remote gateway's public IP from your local gateway? If not, the issue is at the ISP level or a firewall blocking the tunnel establishment itself.
2. Check IKE Phase 1 Logs: Look at your firewall logs. If you see "No proposal chosen," it means your IKE settings (encryption, hash, DH group) do not match.
3. Check IKE Phase 2 Logs: If Phase 1 succeeds but the tunnel won't pass traffic, Phase 2 is failing. This is usually due to a mismatch in the "Proxy IDs" or "Traffic Selectors" (the source/destination subnets).
4. Validate Routing: If the tunnel is up and you can ping the remote gateway IP, but not the remote VMs, check the routing tables. Remember that a tunnel is a transport; it doesn't automatically know where to send traffic once it arrives at the other side.
5. Check MTU/MSS: If you can ping small packets but not large ones, you have an MTU issue. Drop your MSS clamping value and test again.

Summary and Key Takeaways

Configuring a Site-to-Site VPN is a foundational task for building a hybrid network. By bridging your on-premises infrastructure with the cloud, you gain the ability to scale while maintaining control over your internal resources. However, the reliability and security of this connection depend entirely on your attention to detail.

Key Takeaways:

• Design for Stability: Always prefer route-based VPNs over policy-based ones, and implement BGP for dynamic routing to ensure your network handles changes gracefully.
• Prioritize Security: Use IKEv2, strong encryption (AES-256), and high-strength Diffie-Hellman groups. Never use legacy, insecure ciphers.
• Avoid Overlap: Thoroughly map your IP address space before you start. Overlapping CIDR blocks are the single most common cause of permanent, unfixable routing issues in hybrid networks.
• Monitor and Alert: A VPN tunnel should never be a "set it and forget it" component. Monitor its health continuously and set up automated alerts for downtime.
• Embrace Zero Trust: Do not treat the VPN tunnel as a "trusted" zone. Always apply granular security rules to the traffic flowing through the tunnel, just as you would for public-facing traffic.
• Document Everything: Maintain a clear record of your IKE/IPsec parameters. When you need to troubleshoot a connection at 3:00 AM, having a documented "source of truth" will save you hours of frustration.
• Plan for Redundancy: If the connection is business-critical, build for failure. Use multiple tunnels and automated failover mechanisms to ensure that your hybrid architecture remains resilient to individual link failures.

By following these principles, you will move beyond basic connectivity and build a robust, professional-grade hybrid network that supports your organization's growth and operational needs. Remember that the goal is not just to get the tunnel "up," but to create a reliable, secure highway for your data that you can manage with confidence.