Private Mode facilitates Aviatrix deployments without relying on public IPs. Private Mode was introduced on Aviatrix software version 6.8.

Constraints

• AWS and Azure are currently supported.
• Private Mode will not work if you already have gateways deployed in your Controller(Public IPs).
• BGP over LAN functionality is not available.
• Features on the Controller Security tab are not supported.
• FQDN Gateway functionality is unavailable.
• Creation of VPN or Public Subnet Filtering Gateways is not supported.
• Enabling internet-bound egress traffic for inspection through Firewall is not possible.
• Distributed Cloud Firewall (DCF) is not supported.
• Egress for Transit FireNet is not supported.

Architecture

Transit to Spoke data plane tunnels will utilize orchestrated native peering as an underlay. Cloud instances will only have private IPs (Aviatrix Controller, Gateways, and CoPilot) and management traffic occurs through native cloud constructs like Load Balancers, Private Link Services, and peering connections, serving as the foundation for the Aviatrix encrypted Transit network.

Load Balancers

Elastic Load Balancing (ELB) automatically disperses incoming traffic among multiple targets, including EC2 instances, containers, and IP addresses, across one or more Availability Zones. It continuously monitors the health of registered targets and directs traffic solely to those deemed healthy.

Private Link

PrivateLink is a feature that enables communication between customer applications and AWS services using private IP addresses.

Traffic between a VPC endpoint and an endpoint service stays within the AWS network, without traversing the public internet.

Traffic from your VPC is sent to an endpoint service using a connection between the VPC endpoint and the endpoint service.

The following diagram provides a high-level overview of how AWS PrivateLink works:

The user of a service is a service consumer. Service consumers create interface VPC endpoints to connect to endpoint services that are hosted by service providers.

A service provider creates an endpoint service to make their service available in a Region. There are multiple types of VPC endpoints:

• An endpoint network interface refers to a network interface managed by the requester, serving as an ingress point for traffic directed towards an endpoint service. When you create a VPC endpoint and designate specific subnets, AWS generates an endpoint network interface within each specified subnet.
• To establish connectivity between a fleet of virtual appliances utilizing private IP addresses and your VPC, you can create a Gateway Load Balancer endpoint. This endpoint serves as a means to direct traffic towards the fleet. The routing of traffic from your VPC to the Gateway Load Balancer endpoint is managed through route tables.
• A Gateway endpoint is established to facilitate the routing of traffic towards Amazon S3 or DynamoDB. Gateway endpoint does not use private link.

The expected Link Service configuration for AWS is having one Link Service in each region where you want to launch gateways.

It is not possible to launch gateways in the same VPC/VNet as the Link Service VPC/VNet.

VPC Peering

A VPC peering connection establishes a network connection between two Virtual Private Clouds (VPCs), allowing the routing of traffic between them using private IPv4 or IPv6 addresses.

This enables instances within either VPC to communicate as if they were part of the same network.

The need of a Web Proxy

The Controller requires outbound internet access for updates and licensing. AVX Controller expects the proxy to listen on port 3128 and this proxy must be in the same VPC as the controller and copilot.

Squid on Ubuntu Install

To install Squid on Ubuntu, you can follow these steps:

1. Update the package index:

sudo apt update

2. Install Squid using the following command:

sudo apt install squid

3. make sure under “/etc/squid/squid.conf” file the following is configured:

http_access allow localhost
http_access allow localnet
http_port 3128

3. You can start the Squid service using the following command:

sudo systemctl start squid

4. If you want Squid to start automatically every time the system boots, you can enable it as a systemd service:

sudo systemctl enable squid

5. To verify that Squid is running, you can use the following command:

sudo systemctl status squid

By default, Squid listens on port 3128 for incoming proxy connections.

IAM Policy Requirement

The following permissions are required for Private Mode:

{
        "Action": [
                "elasticloadbalancing:DescribeTargetHealth",
                "ec2:CreateVpcEndpointServiceConfiguration",
                "ec2:DeleteVpcEndpointServiceConfigurations",
                "ec2:CreateVpcEndpoint",
                "ec2:DeleteVpcEndpoints",
                "ec2:ModifyVpcEndpointServicePermissions",
                "ec2:DescribeVpcEndpointServicePermissions",
                "ec2:DescribeVpcEndpoints"
        ],
        "Resource": "*",
        "Effect": "Allow"
}

CoPilot Deployment

CoPilot is deployed from marketplace, controller, or using terraform as before.

Configuration

Proxy

Proxy can be configured during the controller initialization:

Private Mode

In the Aviatrix Controller interface, you can access the “Settings” section to configure various features, including “Private Mode.” Change the status from Disabled to Enable:

The Controller will display the message below if the feature was enabled successfully.

The next step is to create the intra-cloud link service. The Intra-Cloud Link Service is configured in the same Cloud as Controller and CoPilot. The Link Service registers the Controller and CoPilot as targets on different ports.

The Controller creates a new subnet (avx-<vpc name>-frontend) in each AZ as showed below:

End Point service is created:

With following NLB:

With the following listeners:

• TCP:31283 (Netflow)
• TCP:443 (Controller)
• TCP:5000 (rsyslog)

CoPilot

CoPilot association can be done from the Private Mode Workflow (this step adds CoPilot to the target groups created before):

But also the association from Settings -> CoPilot is required:

Do not forget CoPilot also will require proxy that is configured under CoPilot Settings -> Configuration -> Proxy Server (protocol needs to be specified):

Transit

Useful tools “Create a VPC” wizard has a option to create only private mode subnet:

The result is show in the screen capture below:

Gateways

I’m going to repeat the process above for a couple of spoke gateways for testing:

The subnets and route tables are show in the figure below:

The spoke creation wizard has a new box for the Cloud Link Service VPC:

Endpoints are created on each VPC in the same subnet where gateways are deployed:

Attachment

The spoke attachment creates the required vpc peerings for the data plane:

Tunnels are established using private IP address.

Testing

For testing purposes, two EC2 instances were deployed as show below:

From Spoke 2 EC2 instance we can ping Spoke 3 EC2 instance:

VPN

If you need help setting up a VPN to the private environment you can use AWS Client VPN EndPoint among other possible solutions.

AWS Client VPN: https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/cvpn-getting-started.html

Mutual authentication: https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/mutual.html

Add the client certificate and key information (mutual authentication): https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/cvpn-working-endpoint-export.html

Deployment Flow

VPN Client VPC -> VPN Client VPC Subnets -> VPN -> Mgmt VPC -> Mgmt VPC Subnets -> VPN Client VPC and Mgmt VPC Peering -> VPN Client VPC and Mgmt VPC Peering routes -> AVX Controller -> Proxy -> Private Mode -> CoPilot

Troubleshooting

• Controller requires public ip address and internet access

References

https://docs.aviatrix.com/documentation/latest/controller-platform-administration/private-mode.html

https://docs.aws.amazon.com/elasticloadbalancing/latest/userguide/what-is-load-balancing.html

https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html

https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html

http://www.squid-cache.org/