Using Aviatrix for Google Cloud VMware Services Secure Cloud Networking Part 1

Google Cloud VMware Engine

VMware Engine is a Google managed VMware software-defined data center (SDDC). It consists of the following components: 

  • VMware vSphere
  • vCenter Server
  • vSAN
  • NSX-T
  • HCX (application migration and workload re-balancing across data centers and clouds)
Diagram from https://cloud.google.com/architecture/private-cloud-networking-for-vmware-engine

VMWare Engine is not available in all regions. Check the regions where the service is available following the link below:

https://cloud.google.com/about/locations/#americas

I’m going to use central-1 for my tests.

Configuration

The first step is enable the VMware Engine API:

Once the API is enable, you have access to the engine dashboard:

I’m going to create a private cloud for testing with a single node:

  • vSphere and vSAN: It will be divided into additional subnets for management, vMotion, vSAN, and NSX (/24 is the minimum)
  • HCX: (/26 or /27)
  • (Gateway) Edge Services: required for client VPN and internet access (/26)
  • Client subnet: assigned to clients connecting via VPN (/24)
  • Workload subnets: configured in NSX-T.
Management subnets.
Diagram from https://cloud.google.com/architecture/private-cloud-networking-for-vmware-engine

GCP alerts you, sending an email, when the private cloud is ready:

Summary of the new created private cloud:

Private Connection

Private services access is a private connection between a custom VPC and a network owned by Google or a third party. VMWare is known as service producer in this case. The private connection establishes a VPC Network Peering connection between a custom VPC and the service producer’s network.

Private connections are a one-to-one relationship between a custom VPC network and a service producer.

We have to allocate address ranges for the private service connection, cloud management, and for workload network segments:

The minimum size is a /24 block

I’ll allocate a /21 for the VMWare services on my vpc where I have my Aviatrix avx-spoke60-us-central1 configured (i fat fingered the ip range: instead of spoke60 I typed spoke50):

The next step is to create the connection(s):

We add a private connection on the Google Cloud Vmware Engine Console. Using the information from the Google Console project dashboard and from vpc peering we can create the connection on the Vmware Engine side:

It can take up to a few minutes for the peer to connect:

Routing

By default, the vpc peering does not import and export routes. We have to edit the vpc peering and select import/export custom routes under the exchange custom routes:

Box checked, click save. The routes learned from the VPC are exported to the VMware Services:

  • imported routes
  • exported routes

The maximum number of routes that a private cloud can receive is 200. The maximum number of routes a private cloud can advertise to a single custom vpc is 100 .

Aviatrix Routing

The diagram illustrates the environment discussed in this post:

The steps to build the FireNet are described here:

Because we need to advertise the GCVE routes learned from the peering connection into the fabric, we create a “Customize Spoke Advertised VPC CIDRs” configuration on the spoke60 gateway:

Once the configuration is saved, we can visualize if it is working checking the spoke50 gateway routes:

  • 192.168.240.0/21 and 192.168.248.0/21 are present

Testing

From NSX-T Manager I created a new segment called 240 to hold a VM I’m going to use to test my deployment:

A VM (vm-1) running Centos is deployed into segment 240 to provide an endpoint and tool set for testing (192.168.240.11):

Egress

There are two possible egress flows:

  • egress through Edge Services (1) (not covered on this blog)
  • egress through Private Service Access (2) and (3)

For flows (2) and (3) the following are mandatory:

  • remove allocated public IP addresses and point-to-site VPN gateways
  • advertise the default route (0.0.0.0/0) from on-prem or from customer vpc
  • enable VPC Service Controls on the VPC peering connection between your VPC network and VMware Engine:
ricardotrentin@RicardontinsMBP % gcloud services vpc-peerings enable-vpc-service-controls --network=avx-bgpolan-us-central1 --service=servicenetworking.googleapis.com
Operation "operations/evsc.p28-491625637231-13c6f190-cde9-4990-ad6b-543c134d3568" finished successfully.
Diagram from https://cloud.google.com/architecture/private-cloud-networking-for-vmware-engine

Ingress

There are two possible ingress flows:

  • P2S VPN, Internet Gateway, and Public IP Service through Edge Services (1)
  • Customer VPC access through Private Service Access (2 and 3 are equivalent)
Diagram from https://cloud.google.com/architecture/private-cloud-networking-for-vmware-engine

Flows (2) and (3) will be ingested through Aviatrix. The ingress flow using Aviatrix and GCP is covered in details here:

Testing

From spoke50 vm I can ping the vm-1 running on GCVE:

Management Access using P2S

Regional Settings

We need to enable Internet Access and Public IP Service which is a requirement for deploying a VPN Gateway we will use to access the environment.

VPN Gateway

This is an OpenVPN-based client VPN that will allow us to connect to the SDDC’s vCenter and perform configurations:

Download the OpenVPN configuration:

The zip file contains the OpenVPN profile and certificates:

Once the profile is loaded and the VPN established:

We can manage the environment through:

  • VCenter:
  • NSX-T

VCenter and NSX Manager information is obtained from the VSphere Management network tab:

Users and password are available on the Summary tab:

Name Resolution for VCenter and NSX-T

https://cloud.google.com/vmware-engine/docs/networking/howto-dns-on-premises

References

https://blogs.vmware.com/cloud/2021/05/18/intro-google-cloud-vmware-engine-deploying-gcve-sddc-hcx/

Leave a Reply