Azure AppGW Stateful Inspection using Aviatrix FireNet

Azure Application Gateway

Azure Application Gateway (AppGW) is a web traffic load balancer that enables you to manage traffic to your web applications. Application Gateway can make routing decisions based on additional attributes of an HTTP request.

For a list of features supported by the AppGW access the link below:

https://docs.microsoft.com/en-us/azure/application-gateway/features

Aviatrix Overview

Aviatrix is a cloud network platform that brings multi-cloud networking, security, and operational visibility capabilities that go beyond what any cloud service provider offers. Aviatrix software leverages AWS, Azure, GCP and Oracle Cloud APIs to interact with and directly program native cloud networking constructs, abstracting the unique complexities of each cloud to form one network data plane, and adds advanced networking, security and operational features enterprises require.

FireNet

Aviatrix Transit FireNet allows the deployment of 3rd party firewalls onto the Aviatrix transit architecture.

Transit FireNet works the same way as the Firewall Network where traffic in and out of the specified Spoke is forwarded to the firewall instances for inspection or policy application.

The topology I’m going to deploy to test is depicted in the diagram below:

a) dedicated vnet for ingress: AppGW is deployed in the spoke30 and backend is located in spoke40

b) shared vnet using subnet inspection: AppGW and backend are in the same vnet (spoke30)

Public and Private subnets are Aviatrix connotation related to the route table associated to a subnet: public subnets have a 0.0.0.0/0 pointing to the internet while private subnets have a 0.0.0.0/0 route entry pointing to None.

Here are the terraform files I used to deploy the test environment:

provider

	terraform {
	required_providers {
	aviatrix = {
	source = "AviatrixSystems/aviatrix"
	version = "2.21.2"
	}
	azurerm = {
	source = "hashicorp/azurerm"
	version = "3.1.0"
	}
	http = {
	source = "hashicorp/http"
	version = "2.1.0"
	}
	}
	}
	provider "aviatrix" {
	controller_ip = var.controller_ip
	username = var.username
	password = var.password
	skip_version_validation = true
	verify_ssl_certificate = false
	}
	provider "azurerm" {
	subscription_id = var.subscription_id
	client_id = var.client_id
	client_secret = var.client_secret
	tenant_id = var.tenant_id
	}
	provider "http" {
	# Configuration options
	}
	data "http" "ip" {
	url = "https://ifconfig.me/"
	}

view raw

gistfile1.txt

hosted with ❤ by GitHub

main

	module "mc_transit" {
	source = "terraform-aviatrix-modules/mc-transit/aviatrix"
	version = "v2.0.0"
	cloud = var.cloud
	cidr = var.vpcs["firenet"]
	region = var.region
	account = var.account
	enable_transit_firenet = true
	lan_cidr = var.vpcs["lan"]
	}

	module "firenet_1" {
	source = "terraform-aviatrix-modules/mc-firenet/aviatrix"
	version = "1.0.0"
	transit_module = module.mc_transit
	firewall_image = var.firewall_image
	firewall_image_version = var.firewall_image_version
	bootstrap_bucket_name_1 = var.storage_bucket_name
	egress_cidr = var.vpcs["egress"]
	egress_enabled = true
	inspection_enabled = true
	instance_size = var.instance_size
	mgmt_cidr = var.vpcs["mgmt"]
	password = var.password
	}
	module "mc-spoke" {
	for_each = {
	"spoke30" = "spoke30"
	"spoke40" = "spoke40"
	"ingress" = "ingress"
	}
	source = "terraform-aviatrix-modules/mc-spoke/aviatrix"
	version = "1.1.2"
	account = var.account
	cloud = var.cloud
	name = "az-${each.value}-${var.region}"
	region = var.region
	cidr = var.vpcs["${each.value}"]
	inspection = true
	transit_gw = module.mc_transit.transit_gateway.gw_name
	ha_gw = true
	instance_size = var.instance_size
	single_az_ha = false
	}

view raw

gistfile1.txt

hosted with ❤ by GitHub

For details on how to deploy an Aviatrix Firenet with PANs on Azure please access the link below:

https://docs.microsoft.com/en-us/azure/application-gateway/features

AppGW Deployment

The AppGW components are show in the diagram below:

V1 and V2

Application Gateway and Web Application Firewall (WAF) v2 offers additional features such as auto-scaling and availability-zone redundancy.

Constraints

A dedicated subnet is required for the application gateway. You can have multiple instances of a given application gateway deployment in a subnet. You can also deploy other application gateways in the subnet.
You can’t mix Standard_v2 and Standard Azure Application Gateway on the same subnet.
Although a /24 subnet is not required per Application Gateway v2 SKU deployment, it is highly recommended. This is to ensure that Application Gateway v2 has sufficient space for autoscaling expansion and maintenance upgrades.
Any scenario where 0.0.0.0/0 needs to be redirected through any virtual appliance, a hub/spoke virtual network, or on-premises (forced tunneling) isn’t supported for V2.
You must allow incoming Internet traffic on TCP ports 65503–65534 for the Application Gateway v1 SKU, and TCP ports 65200–65535 for the v2 SKU with the destination subnet as Any and source as GatewayManager service tag. This port range is required for Azure infrastructure communication.
Outbound Internet connectivity can’t be blocked. Default outbound rules in the NSG allow Internet connectivity. We recommend that you:
Don’t remove the default outbound rules.
Don’t create other outbound rules that deny any outbound connectivity.
Traffic from the AzureLoadBalancer tag with the destination subnet as Any must be allowed.
Service endpoint are currently not supported in an Application Gateway subnet.

Deployment

The first scenario the Back End is located in the vnet spoke 40:

Testing

Browning the Front End IP of the AppGW properly displays the nginx default page for Centos 7:

A tcpdump from the backend VM shows the AppGW requests reaching out properly as the health probes coming from Azure:

AppGW scale out creates two instances when the minimum amount is equal to 1. If we increase the number of instances we will see more IPs on the logs and captures.

From CoPilot FlowIQ we can see those flows:

From one of the firewalls in the Firenet we can see the flow is inspected:

Subnet Inspection

Aviatrix’s subnet inspection feature allows you to redirect subnet-level traffic to the Aviatrix Transit FireNet Gateway for inspection by a next-generation firewall (NGFW) appliance. With this feature subnets are assigned to a subnet group and specify an inspection policy for the subnet group. Traffic between subnets in different subnet groups with an inspection policy is redirected to the NGFW for inspection.

Aviatrix version 6.6.c and 6.7 are the minimum recommended for Subnet Inspection as it brings flows optimization

Subnet Group

A subnet group defines a logical grouping of subnets in a VNet. A subnet group is local to a VNet and does not span across multiple VNets.

Contraints

Egress FireNet Inspection (Dual Transit FireNet)
There is a downtime of 10–20 seconds when you add or remove subnets from a subnet group
Customized SNAT on Spoke gateway
Customized Spoke advertised CIDRs
Customized Spoke VPC Route Table
Filter Learned Routes to Spoke VPC
Auto-advertised Spoke Site2Cloud CIDRs
BGP on Spoke

Configuration

Subnet Firenet Inspection is configured under Multi-Cloud Transit List selecting the desired spoke and clicking on the actions drop-down menu:

I’ll create two subnet groups (to inspect traffic between subnets in the same VNet, the subnets must be in different subnet groups): one for the AppGW and other for the back end VM(s):

spoke30-subnetgroup for the back end(s)

to inspect traffic between subnets in the same VNet, the subnets must be in different subnet groups

spoke30appgw for the AppGW

Once the subnet groups are created and populated, we have to enable the inspection policy:

Checking changes to the routes and route tables:

AppGW: both public subnets added to the same group are reachable because route entries pointing to the virtual network were created for them. Everything else is directed to the spoke GW:

Back end VM: both private subnets added to the same group are reachable because route entries pointing to the virtual network were created for them. Everything else is directed to the spoke GW: