Using Aviatrix for Google Cloud VMware Services Secure Cloud Networking Part 1

Google Cloud VMware Engine

VMware Engine is a Google managed VMware software-defined data center (SDDC). It consists of the following components:

VMware vSphere
vCenter Server
vSAN
NSX-T
HCX (application migration and workload re-balancing across data centers and clouds)

Diagram from https://cloud.google.com/architecture/private-cloud-networking-for-vmware-engine

VMWare Engine is not available in all regions. Check the regions where the service is available following the link below:

https://cloud.google.com/about/locations/#americas

I’m going to use central-1 for my tests.

Configuration

The first step is enable the VMware Engine API:

Once the API is enable, you have access to the engine dashboard:

I’m going to create a private cloud for testing with a single node:

vSphere and vSAN: It will be divided into additional subnets for management, vMotion, vSAN, and NSX (/24 is the minimum)
HCX: (/26 or /27)
(Gateway) Edge Services: required for client VPN and internet access (/26)
Client subnet: assigned to clients connecting via VPN (/24)
Workload subnets: configured in NSX-T.

Diagram from https://cloud.google.com/architecture/private-cloud-networking-for-vmware-engine

GCP alerts you, sending an email, when the private cloud is ready:

Summary of the new created private cloud:

Private Connection

Private services access is a private connection between a custom VPC and a network owned by Google or a third party. VMWare is known as service producer in this case. The private connection establishes a VPC Network Peering connection between a custom VPC and the service producer’s network.

Private connections are a one-to-one relationship between a custom VPC network and a service producer.

We have to allocate address ranges for the private service connection, cloud management, and for workload network segments:

The minimum size is a /24 block

I’ll allocate a /21 for the VMWare services on my vpc where I have my Aviatrix avx-spoke60-us-central1 configured (i fat fingered the ip range: instead of spoke60 I typed spoke50):

The next step is to create the connection(s):

We add a private connection on the Google Cloud Vmware Engine Console. Using the information from the Google Console project dashboard and from vpc peering we can create the connection on the Vmware Engine side:

It can take up to a few minutes for the peer to connect:

Routing

By default, the vpc peering does not import and export routes. We have to edit the vpc peering and select import/export custom routes under the exchange custom routes:

Box checked, click save. The routes learned from the VPC are exported to the VMware Services:

imported routes

exported routes

The maximum number of routes that a private cloud can receive is 200. The maximum number of routes a private cloud can advertise to a single custom vpc is 100 .

Aviatrix Routing

The diagram illustrates the environment discussed in this post:

The steps to build the FireNet are described here:

Deploying an Aviatrix FireNet on GCP with PANs

Because we need to advertise the GCVE routes learned from the peering connection into the fabric, we create a “Customize Spoke Advertised VPC CIDRs” configuration on the spoke60 gateway:

Once the configuration is saved, we can visualize if it is working checking the spoke50 gateway routes:

192.168.240.0/21 and 192.168.248.0/21 are present

Testing

From NSX-T Manager I created a new segment called 240 to hold a VM I’m going to use to test my deployment:

A VM (vm-1) running Centos is deployed into segment 240 to provide an endpoint and tool set for testing (192.168.240.11):

Egress

There are two possible egress flows:

egress through Edge Services (1) (not covered on this blog)
egress through Private Service Access (2) and (3)

For flows (2) and (3) the following are mandatory:

remove allocated public IP addresses and point-to-site VPN gateways
advertise the default route (0.0.0.0/0) from on-prem or from customer vpc
enable VPC Service Controls on the VPC peering connection between your VPC network and VMware Engine:

ricardotrentin@RicardontinsMBP % gcloud services vpc-peerings enable-vpc-service-controls --network=avx-bgpolan-us-central1 --service=servicenetworking.googleapis.com
Operation "operations/evsc.p28-491625637231-13c6f190-cde9-4990-ad6b-543c134d3568" finished successfully.