SDN+NFV

vRouter HA in AWS across Availability Zones

by aannavarapu ‎06-04-2016 03:47 PM - edited ‎06-09-2016 10:00 AM (6,281 Views)

vRouter HA in AWS across Availability Zones

 

 

vRouter can be used as a gateway in the AWS VPC to handle Routing, Firewall, VPN, NAT etc. To eliminate a single point of failure and have a high availability solution, a secondary vRouter can be deployed as the standby. Traditional solutions like VRRP cannot be used as the AWS VPC does not support multi-cast traffic. Moreover, the virtual IP is not an L2 adjacent IP address but rather the address of the Igw. As such VRRP based solution will not work on AWS.

 

A script based solution to address the HA scenario is apt in this case. Two vRouters running in a VPC across two different Availability zones and with GRE tunnel configured between them is the basis for this solution. The following topology highlights the architecture for a HA deployment of a network architecture using vRouter in AWS.

 

aws-ha-vrouter.png

 

Single VPC architecture with 2 availability zones – Deployment procedure for vRouter HA

 

 The following is the step-by-step procedure for enabling HA deployment of vRouter in AWS.

 

  • Create an IAM role with a custom policy to execute select Amazon EC2 API calls. For access to more APIs, include them in the policy. The following policy is sufficient for making VPC routing table edits.
   

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:associateRouteTable",
"ec2:createRoute",
"ec2:createRouteTable",
"ec2:deleteRoute",
"ec2:deleteRouteTable",
"ec2:describeRouteTables",
"ec2:describeVpcs",
"ec2:replaceRoute",
"ec2:disassociateRouteTable",
"ec2:replaceRouteTableAssociation"
],
"Resource": "*"
}
]
}

 

  • Create vRouter instances in the two Availability zones (AZ1 and AZ2) using the IAM role.

 

  • Split the VPC route table into a public and private routing tables. Configure the default route of the public routing table to be the igw and the default route of the private routing table to the network interface of the vRouter.

 

  • Create a GRE tunnel (tun0 in this example) between the two routers (vRouter-active and vRouter-standby). This tunnel is used to understand the health of each other using BFD. 10.3.3.0/30 is the subnet used in this example for the GRE tunnel interfaces. Local-ip and remote-ip configuration in the gre tunnel represent the ip addresses of the vRouter interfaces.

 

vRouter-active

vRouter-standby

GRE configuration commands

set interfaces tunnel tun0 address 10.3.3.1/30

 

set interfaces tunnel tun0 encapsulation gre

 

set interfaces tunnel tun0 local-ip 172.31.6.36

 

set interfaces tunnel tun0 remote-ip 172.31.19.209

set interfaces tunnel tun0 address 10.3.3.2/30

 

set interfaces tunnel tun0 encapsulation gre

 

set interfaces tunnel tun0 local-ip 172.31.19.209

 

set interfaces tunnel tun0 remote-ip 172.31.6.36

 

 

  • Configure BFD to monitor the tunnel interface. BFD configuration on vRouter takes a template (template name is aws in our example) with timers for rx and tx of BFD messages and a multiplier that tells how many messages to wait before classifying a status as failure. In our example, we used 300 ms as the rx and tx with a multiplier of 3 which means that link detection can occur in 900ms. BFD messages can be authenticated with passwords. Once the template is configured, define the bfd interfaces and attach the template.

 

vRouter-active

vRouter-standby

BFD configuration commands

set protocols bfd template aws auth simple key ********

 

set protocols bfd template aws minimum-rx 300

 

set protocols bfd template aws minimum-tx 300

 

set protocols bfd template aws multiplier 3

 

set protocols bfd destination 10.3.3.2 source 10.3.3.1 template aws

set protocols bfd template aws auth simple key ********

 

set protocols bfd template aws minimum-rx 300

 

set protocols bfd template aws minimum-tx 300

 

set protocols bfd template aws multiplier 3

 

set protocols bfd destination 10.3.3.1 source 10.3.3.2 template aws

 

 

  • Configure routing protocol (static routing in this example) that acts as the client protocol for BFD. For this purpose, define a loopback interface and advertise it via the GRE tunnel to the other vRouter using routing protocols (static in our example) and the routing protocol will be the client for BFD to start monitoring. 

 

vRouter-active

vRouter-standby

Interface & routing configuration commands

set interfaces loopback lo1 address 1.1.1.1/32

 

set protocols static route 2.2.2.2/32 next-hop 10.3.3.2 fall-over bfd

set interfaces loopback lo1 address 2.2.2.2/32                

 

set protocols static route 1.1.1.1/32 next-hop 10.3.3.1 fall-over bfd

 

  • Verify the bfd state using “show bfd session” and check if the sess-state is “Up”

 

 

 

vRouter-active

vyatta@vRouter-active:~$ show bfd session

Sess-Idx   Remote-Disc  Lower-Layer  Sess-Type   Sess-State  UP-Time   Remote-Addr

1          1            IPv4         Single-Hop  Up          02:22:16  10.3.3.2/32

vRouter-standby

vyatta@vRouter-standby:~$ show bfd session

Sess-Idx   Remote-Disc  Lower-Layer  Sess-Type   Sess-State  UP-Time   Remote-Addr

1          1            IPv4         Single-Hop  Up          02:24:28  10.3.3.1/32

 

 

  • Give superuser access to the vRouter login user. Scp the java tar file into the vRouter and install Java JRE on the vRouter. The example below shows the vRouter-standby.

 

 

vyatta@vRouter-standby:~$ set system login user vyatta level superuser

vyatta@vRouter-standby:~$ mkdir /usr/java

vyatta@vRouter-standby:~$ sudo chmod 755 /usr/java

vyatta@vRouter-standby:~$ sudo cp jre-8u74-linux-x64.tar /usr/java/

vyatta@vRouter-standby:~$ sudo tar -xvf jre-8u74-linux-x64.tar

 

 

  • Configure the environment variables necessary to run the Amazon API calls (make it persistent across reloads by updating the /etc/profile file).

 

 

vyatta@vRouter-standby:~$ vi /etc/profile

export JAVA_HOME="/usr/java/jre1.8.0_74"

export PATH="$PATH:/usr/bin"

export PATH="$PATH:$JAVA_HOME/bin"

export EC2_HOME="/usr"

 

 

  • Update the LIBDIR location in /usr/bin/ec2-cmd. Hash the old location.

 

 

vyatta@vRouter-standby:~$ vi /usr/bin/ec2-cmd

#LIBDIR="$ {EC2_HOME}/lib"

LIBDIR="${EC2_HOME}/lib/ec2-api-tools"

 

 

  • Check if the vRouter is ready to execute Amazon API commands 

 

 

vyatta@vRouter-standby:~$ ec2-describe-vpcs

VPC    vpc-f9aca99d  available     172.31.0.0/16 dopt-41580424 default       true

 

  • Place the following python script in /config folder such that upgrades will not delete the file. Edit the script based on your architecture. Comments in the script guide the needed values.

 

More optimizations can be done on the script like checking the existing route table entry by using the command “ec2-describe-route-tables <rtd-id> --region <region>” and only updating if the route does not already exist (with the same interface). “-D” option in the ec2-cmd is a Dry-run issue of the command and will not make changes.

 

 

 

import vyatta.configd as configd

from sys import stderr, stdout

from os import getpid

import os

import time

 

state = "Down" # State of the interface on the other side of the GRE tunnel

remote_ip = "10.3.3.1/32" # IP address of the remote GRE tunnel interface

 

vpc_router_id = "rtb-a7b8edc2" # Router ID of the VPC route table (private routing table) that need to be changed

region = "us-west-1" # Region in which the VPC exists

nid = "eni-755c7712" # network interface id of the standby vRouter as the default gateway for private subnet (this)

cidr = "0.0.0.0/0" # network to change the routing table entry for

 

def main():

    client = configd.Client()

    client.session_setup(str(getpid()))

    while True:

        try:

                command_args = "bfd session"

                data = client.call_rpc_dict("vyatta-opd-v1","command",{"command":"show", "args":command_args})["output"]

                lines = data.split('\n')

                columns = lines[1].split('\t')

                data = columns[0].split()

                ec2_cmd = 'ec2-replace-route %s --region %s -v -n %s -r %s' % (vpc_router_id, region, nid, cidr)

                if data[4] == state and data [6] == remote_ip:

                        f = os.popen(ec2_cmd)

                        now = f.read()

                        print now

        except configd.Exception as e:

                stderr.write(str(e))

                client.session_teardown()

        time.sleep(60) #change the timer as needed

 

if __name__ == "__main__":

    main()

 

 

  • Execute the script using “python change-ec2-route.py &” or edit the /etc/rc.local file and enter the execution command in the file to make sure that the script execution persists across reloads.

 

 

vyatta@vRouter-standby:~$ sudo vi /etc/rc.local

#!/bin/sh -e

#

# rc.local

#

# This script is executed at the end of each multiuser runlevel.

# Make sure that the script will "exit 0" on success or any other

# value on error.

#

# In order to enable or disable this script just change the execution

# bits.

#

# By default this script does nothing.

python /config/change-ec2-route.py&

exit 0

 

 

 

 

Contributors