The diagram below shows the environment I’m going to test:

Active-Standby

This option supports connecting AVX transit gateways to on-prem with only one active tunnel and the other one as backup. The use case is a deployment scenario where on-prem device such as firewall does not support asymmetric routing on two tunnels. Aviatrix configuration:

• ECMP is disabled
• Active-Standby is enabled

The active/standby configuration will produce the following configuration:

• transit-us-east-1 is the primary path with a metric of 100
• transit-us-east-1 to transit-us-east-1-hagw has a metric of 200
• transit-us-east-1-ha is the backup path with a metric of 300

FortiGate config

To align the FortiGate configuration to the AVX gateways, we need to use BGP Weight attribute to prefer a route received from the AVX primary transit gateway GRE tunnel over the AVX transit gateway ha GRE tunnel.

GRE:

edit "toAVX"
    set vdom "root"
    set ip 169.254.102.229 255.255.255.255
    set allowaccess ping
    set type tunnel
    set remote-ip 169.254.102.230 255.255.255.252
    set interface "port1"
next

edit "toAVX-HA"
    set vdom "root"
    set ip 169.254.62.33 255.255.255.255
    set allowaccess ping
    set type tunnel
    set remote-ip 169.254.62.34 255.255.255.252
    set interface "port1"
end

BGP:

config router bgp
    set as 65500
    set router-id 192.168.1.1
    config neighbor
        edit "169.254.102.230"
            set soft-reconfiguration enable
            set remote-as 65501
            set update-source "192.168.1.1"
        next
        edit "169.254.62.34"
            set soft-reconfiguration enable
            set remote-as 65501
            set update-source "192.168.1.1"
        next
    end
    config redistribute "connected"
        set status enable
    end
end

BGP traffic engineering:

config router prefix-list
    edit "prf-10.0.0.0-8"
        config rule
            edit 2
                set prefix 10.0.0.0 0.0.0.0
                set ge 8
                unset le
            next
        end
    next
end

config router route-map
    edit "rt-map-10.0.0.0-8"
            # config rule
                edit 1
                    set match-ip-address "prf-10.0.0.0-8"
                    set set-weight 40000
                next
            end
    next
end

config router bgp
    set as 65500
    set router-id 192.168.1.1
    config neighbor
        edit "169.254.102.230"
            set soft-reconfiguration enable
            set remote-as 65501
            set route-map-in "rt-map-10.0.0.0-8"
        next
        edit "169.254.62.34"
            set soft-reconfiguration enable
            set remote-as 65501
        next
    end

Checking

 execute router clear bgp ip 169.254.102.230 soft

get router info bgp network 

Testing

Shooting down the primary AVX gateway “promotes” the “surviving” routes from the AVX HA gateway GRE tunnel to the preferred path and those routes are installed into the FortiGate routing table:

Active/Active

This option supports connecting AVX transit gateways to on-prem with more than one tunnel active. Aviatrix configuration:

• BGP ECMP is enabled
• Active-Standby is disabled

The active/standby configuration will produce the following configuration:

• transit-us-east-1 is the primary path with a metric of 100
• transit-us-east-1 to transit-us-east-1-hagw has a metric of 200
• transit-us-east-1-ha is also a primary path with a metric of 100

FortiGate config

To align the FortiGate configuration to the AVX gateways, we need to configure ECMP:

config router bgp
    set as 65500
    set router-id 192.168.1.1
    set ebgp-multipath enable
    set ibgp-multipath enable
    set additional-path enable
    config neighbor
        edit "169.254.102.230"
            set soft-reconfiguration enable
            set remote-as 65501
        next
        edit "169.254.62.34"
            set soft-reconfiguration enable
            set remote-as 65501
        next
    end
    config redistribute "connected"
        set status enable
    end

Checking

Asymmetric Routing

AVX and FortiGate ECMP enabled causes half of the traffic to get dropped. It happens on flows that takes different paths on egressing and ingressing FortiGate GRE tunnels. If a FortiGate receives the response packets, but not the requests, by default it blocks the packets as invalid. This behavior is known as asymmetric routing.

It is not recommended but if it is required that the FortiGate should permit asymmetric routing, it can be configured with the following command:

config system settings
    set asymroute enable
end

References

List of all useful BGP debug and verification commands:

show router bgp
get router info bgp summary
get router info bgp network
get router info routing-table bgp
get router info bgp neighbors
get router info bgp neighbors advertised-routes
get router info bgp neighbors routes
get router info bgp neighbors received-routes
diagnose sys tcpsock | grep 179
diagnose ip router bgp level info
diagnose ip router bgp all enable
exec router clear bgp all

https://community.fortinet.com/t5/FortiGate/Technical-Tip-BGP-route-selection-process/ta-p/195932

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Configuring-link-redundancy-Traffic-load-balancing/ta-p/193150

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Use-BGP-Weight-attribute-to-prefer-default-route/ta-p/195347

https://community.fortinet.com/t5/FortiGate/Technical-Note-How-the-FortiGate-behaves-when-asymmetric-routing/ta-p/198575