Many of the financial industry clients we have worked with require a large virtual network footprint on AWS. They require complex, highly advanced network architectures that are hybrid (on-premises to AWS), multi-VPC, multi-Account, and cross-regional for a variety of reasons including multi-region applications and disaster recovery.
AWS offers a plethora of services and products each providing a very niche networking functionality. There are many options to consider for even simple functionality such as VPC to VPC connectivity. Having to consider the range of products themselves could be intimidating for a network administrator who must consider the maintenance, availability, architecture, security, and scalability of the company’s AWS Network while ensuring cost efficiency.
This AWS Network Foundation solution provides an opinionated, foundational set of network architectures, design considerations, automated solutions, and best practices that help our clients “quick start” their AWS journey with a strong foothold while ensuring the scalability and growth they are likely to require as their AWS footprint grows.
Vertical Relevance highly recommends that the following foundational best practices be implemented to create a sustainable AWS Network that will support an enterprise-level organization. See our curated list of these best practices here.
Single Account Networks – Three-Tier Virtual Private Cloud (VPC)
This solution provides an overview of basic AWS Networking Components and Infrastructure as Code (IaC) for deploying a VPC that follows the well-established three-tier architecture best practices for an application.
The three-tier architecture is the most popular implementation of a multi-tier architecture and consists of a single presentation tier, logic tier, and data tier. The following illustration shows an example of a simple, generic three-tier application.
Presentation tier
The presentation tier is the user interface layer of the application, where the end-user interacts with the application. Its main purpose is to display information to and collect information from the user. This top-level tier can run on a web browser on AWS. Web presentation tiers are usually developed using HTML, CSS, and JavaScript.
Logic (Backend) tier
The Logic tier the heart of the application running the required business/application logic. Only the presentation tier traffic can directly reach the logic tier. In this tier, information collected in the presentation tier is processed – sometimes against information in the database in the data tier. The logic tier can also add, delete, or modify data in the data tier. The logic tier is typically developed using Python, JavaScript, Java, Perl, PHP, or Ruby, and communicates with the data tier using API calls.
Data Tier
The data tier is where the information processed by the application is stored and managed. This can be a relational database management system such as RDS (PostgreSQL, MySQL, MariaDB, Oracle, Microsoft SQL Server), DB2, or Informix, or in a NoSQL Database server such as DynamoDB, Cassandra, CouchDB, or MongoDB.
This solution demonstrates how we can implement this three-tier architecture on AWS. It also provides details for each of the foundational AWS components used to create this architecture. This will help you understand the core foundations of deploying and various networking modules on AWS.
Components
The following components on AWS form the network building blocks of any application. Having a deep understanding of these foundational networking components is crucial in further building more complex and intertwined networks on AWS.
- Region – An AWS region is a physical location around the world where a cluster of data centers reside.
- Availability Zone (AZs) – Each AWS Region consists of multiple, isolated, and physically separate AZs within a geographic area. Workloads requiring high availability should be run in multiple AZs to achieve fault-tolerance.
- VPC – A logically isolated, virtual network in your AWS account. VPC is the most foundational concept to grasp on AWS from a networking point of view. VPC gives you complete control over your virtual networking environment, including resource placement, connectivity, and security. A VPC spans multiple regions within a single region.
- Subnet – A smaller network which is a subset of IP addresses in your VPC. Each subnet must reside entirely within one Availability Zone and cannot span zones.
- Route Table – A set of rules, called routes, that are used to determine where network traffic is directed. Each subnet requires at least one route table to route the traffic as defined by the route table.
- VPC Router – Each AWS VPC has a VPC router. This VPC router is primarily managed by AWS. However, understanding the functionality of the VPC router helps you better manage all your network traffic. The primary function of the VPC router is to take all the route tables defined within that VPC, and then direct the traffic flow within that VPC, as well as to subnets outside of the VPC, based on the rules defined within those tables.
- NAT Gateway – A NAT gateway is a Network Address Translation (NAT) service. NAT Gateways have a public IP and help keep private instances private. They allow the instances in your private subnet to instantiate and connect to services outside your VPC using NAT IP conversion. However, external services cannot initiate the connection with your private instances.
- Internet Gateway – An Internet Gateway is a VPC component that allows communication between your VPC and the internet. It serves two purposes
- Provide a target in your VPC route tables for internet bound traffic (including traffic from NAT Gateways)
- Perform network address translation (NAT) for instances that have been directly assigned public IPv4 addresses.
- Security Groups and NACLs – Security Groups and Network Access Control Lists (NACLs) act as a virtual firewall to protect your network. They both use sets of inbound and outbound rules to control traffic to and from resources in a VPC. However, Security groups function at the instance level of a VPC. Therefore, each security group can be applied to one or more instances, even across subnets. NACLs control traffic to or from a subnet according to a set of inbound and outbound rules. Network Security best practices for using Security Groups and NACLs should be followed to ensure that the workloads in these VPCs are secure.
How It Works
The diagram below will help visualize the logical networks we are building. The blueprint (codebase) below can be used to create a similar three-tier VPC for your needs.
- VPC Creation When you create a VPC, you must specify a range of IPv4 addresses for the VPC in the form of a Classless Inter-Domain Routing (CIDR) block; for example, 10.0.0.0/16. This is the primary CIDR block for your VPC. We must provide a region and VPC CIDR block as variable inputs to create the VPC.
- At least 2 Availability Zones are provided for high availability. The VPC will span these AZs.
- Within each AZ, three subnets are created Public, Private, and Isolated. A route table is also created and associated with each subnet.
- “Public subnet” means that it has a route to the internet gateway in its associated route table. NAT Gateways with public IPs are also created within this public subnet. One NAT Gateway should be added to each public subnet for high availability. All the front-end workloads/servers must be deployed in the public subnet.
- “Private subnet” means that it has a route to the NAT Gateway in its associated route table. All the backend/application workloads must be deployed in the private subnet.
- “Isolate subnet” means that only resources from private subnet can communicate with the resources in the isolated subnet. Database components must be deployed in the isolated subnet.
- An internet gateway is attached to the VPC. A route with CIDR 0.0.0.0/0 and destination internet gateway is added to the public subnet route tables for internet bound traffic.
Blueprint
The Blueprint contains the product Infrastructure as Code written in AWS CDK using Typescript. The three-tier VPC product combines the above-mentioned components and creates the architecture shown in the above diagram. Further details on how to deploy this three-tier VPC are included in the READEME.md file.
Benefits
The chief benefit of three-tier architecture is its logical and physical separation of functionality. Each tier can run on a separate operating system and server platform – e.g., web server, application server, database server – that best fits its functional requirements. Other benefits include
- Faster development – Because each tier can be developed and managed by different teams, applications can be brought to the market faster. Programmers can use the best languages and tools for each tier meeting their requirements.
- Improved scalability – Any tier can be scaled independently of the others as needed.
- Improved reliability – An outage in one tier is less likely to impact the availability or performance of the other tiers.
- Improved security – Because the presentation tier and data tier can’t communicate directly, a well-designed application tier can function as a sort of internal firewall, preventing SQL injections and other malicious exploits.
End Results
The VPC Design paired with the CIDR management practices mentioned above will give your organization the ability to quickly spin up a solution that application teams can take and quickly start building architectures. It provides a strong foundational component for any AWS Network Architecture.
VPC Peering
VPC Peering feature of AWS provides the ability to allow communication between two VPCs over the private AWS network. This solution provides details of creating VPC peers and codebase that can be used to create these peers. VPC Peering does not provide transitive networking. Therefore, the complexity of managing VPC peers can exponentially increase as the number of VPCs requiring intercommunication grows.
VPC peering connection is a networking connection between two VPCs that enables you to route traffic between them. This traffic remains within the AWS network backbone and does not traverse the internet. Instances in either VPC can communicate with each other as if they are within the same network. You can create a VPC peering connection between your own VPCs, or with a VPC in another AWS account. The VPCs can be in different regions (also known as an inter-region VPC peering connection).
Components
The following components on AWS are created to establish the VPC Peering connection.
- VPC - A logically isolated, virtual network in your AWS account. VPC is the most foundational concept to grasp on AWS from a networking point of view. VPC gives you complete control over your virtual networking environment, including resource placement, connectivity, and security. A VPC spans multiple regions within a single region.
- VPC Peering connection - An AWS component, attached to your VPC, that helps you facilitate the transfer of data. For example, if you have more than one AWS account, you can peer the VPCs across those accounts to create a file sharing network. You can also use a VPC peering connection to allow other VPCs to access resources you have in one of your VPCs.
- Route Table - A set of rules, called routes, that are used to determine where network traffic is directed. Each subnet requires at least one route table to route the traffic as defined by the route table. Once the VPC peering connection is established, the route table of each subnet requiring connection to the peered VPC must be updated with a route that has CIDR of the peered VPC and destination of the peering connection ID.
How It Works
The diagram below will help visualize the logical network we are building. The IaC codebase can be found in the blueprint section below.
- This solution builds two VPCs (as described in the three-tier VPC section) in the same region and establishes the VPC Peering connection between these VPCs. If cross-region or cross-account peering connection is required, please see instructions in the blueprint.
- Once the VPCs and the peering connection are established, the solution also updates the appropriate routes in each VPC subnet route tables to direct traffic destined for the other VPC via the VPC Peering Connection.
Blueprint
The Blueprint contains the product Infrastructure as Code written in AWS CDK using Typescript. With minor modifications, the blueprint can be quickly used to create a similar network architecture and VPC Peering connections to meet your AWS Networking needs. Further details on how to deploy the VPCs and the VPC peering connections are included in the READEME.md file. Instructions on how to add cross-region or cross-account peering connections are also provided.
Benefits
- Establish peering relationships between VPCs across different AWS Regions and accounts. This allows any VPC resource including EC2 instances, RDS databases and Lambda functions that run in different AWS Regions to communicate with each other using private IP addresses, without requiring gateways, VPN connections, or separate network appliances.
- Traffic always stays on the global AWS backbone, and never traverses the public internet, which reduces threats, such as common exploits, and DDoS attacks.
- All traffic is encrypted
- VPC peering connection is highly available with no single point of failure, or bandwidth bottleneck.
- VPC peering is a cost-effective way to share resources between regions or replicate data for geographic redundancy compared to solutions that may include VPN connections or other network appliances.
End Results
A VPC peering connection between your VPCs allows you to establish traffic between different applications and AWS workloads across VPCs in different regions and accounts in a secure, highly available and cost-effective way.
Note, VPC Peering should be used only for testing purposes or for architecture designs where it is confirmed that the applications do not (and will not) require communication with any other VPCs as it is not a scalable solution. For a truly scalable and manageable solution, see the Hub and Spoke Model or AWS Cloud WAN solutions below.
Building a Multi-Account + Multi-Region Network
The three-tier VPC solution described above is a great way to kick start the AWS Networking Journey. It will be the core component of any application and scalable infrastructure running on AWS. However, as the number of applications and workloads on AWS expands, our clients will usually require inter-account and inter-region connectivity.
Connectivity between multiple accounts may be required to allow different apps to communicate over the private network. Inter-region connectivity may be required for applications that span multiple regions.
The following set of solutions offers various ways of establishing this connectivity. Each approach has its own pros and cons that should be considered prior to deploy the solution on a larger scale. Page Break
Hub and Spoke model (Transit Gateway + Network Manager)
As the number of VPCs requiring intercommunication increases, the complexity of managing the VPC Peering connections increases exponentially. This is where Transit Gateway comes into play. AWS Transit Gateway provides a hub and spoke design for connecting thousands of VPCs and on-premises networks while maintaining high availability and scalability.
Components
- VPC – A logically isolated, virtual network in your AWS account.
- VPC Subnet Route Tables – The Subnet Route Tables in charge of directing the flow of traffic originating from the subnet.
- Transit Gateway – A network transit hub that you can use to interconnect your virtual private clouds (VPCs) and on-premises networks. Transit Gateways are regional resources.
- Transit Gateway Attachments – Attachments are required to be added to the VPC to be able to route traffic through to the Transit Gateway.
- Transit Gateway Route Tables – A transit gateway has a default route table and can optionally have additional route tables. A route table includes dynamic and static routes that decide the next hop based on the destination IP address of the packet. The target of these routes could be any transit gateway attachment.
- Transit Gateway Network Manager – A dashboard that provides a single global view of your private network. It allows you to visualize the global network in a topology diagram and on a geographical map. You can review utilization metrics, such as bytes in/out, packets in/out, packets dropped, and alerts for changes in the topology, routing, and up/down connection status, and more easily manage your entire global network.
How it works
The diagram below will help visualize the logical network that we are building. The IaC codebase can be found in the blueprint section below. Note: The solution below creates a single region, single account hub and spoke model. If cross-region or cross-account peering connection is required, please see instructions in the blueprint for Transit Gateway Peering.
- This solution builds three VPCs (as described in the three-tier VPC section) in the same region.
- It creates an AWS Transit Gateway in the same region.
- Transit Gateway attachments are then added to each of the VPCs so that traffic can be routed through to the Transit Gateway.
- Each VPC is added to the default route table of the Transit Gateway so that any traffic routed to the Transit Gateway can be forwarded to the appropriate destination.
- Route tables of the private subnets for each VPC are also updated so that any traffic destined for other VPCs is routed to the Transit Gateway attachment to the Transit Gateway.
Figure-05
Blueprint
The Blueprint contains the product Infrastructure as Code written in AWS CDK using Typescript. With minor modifications, the blueprint can be quickly used to create a similar network architecture and VPC Peering connections to meet your AWS Networking needs. Further details on how to deploy the VPCs and the VPC peering connections are included in the READEME.md file.
Benefits
The Hub and Spoke model described above provides the following benefits:
- Establish connectivity between a large number of VPCs across different AWS accounts in the same region. The Transit Gateway product makes this a highly scalable solution. This allows any VPC resource including EC2 instances, RDS databases, and Lambda functions that run in different AWS Regions to communicate with each other using private IP addresses, without requiring gateways, VPN connections, or separate network appliances.
- Traffic always stays on the global AWS backbone, and never traverses the public internet, which reduces threats, such as common exploits, and DDoS attacks.
- All traffic is encrypted
- Transit Gateway is highly available with no single point of failure, or bandwidth bottleneck.
- Transit Gateway can be used to connect with on-premises networks using solutions such as VPN and Direct Connect. See our VPN at Scale for Financial Services spotlight if you’re interested.
- Hub and Spoke model are more manageable than the VPC Peering solution as the number of connections required increases linearly with the number of resources added.
End Results
The Hub and Spoke model and solution described above provides a truly scalable AWS networks solution for an organization that would require cross-account, cross-region communication. Transit Gateway peering is readily available for cross-region communication. Transit gateway peering unlocks the ability to build a truly global AWS network infrastructure.
However, the solution described above does not provide full automation of network resources The management of maintaining different routing tables and attachments could also be cumbersome for an organization with 10s or 100s of VPCs and requiring hybrid connectivity. The next two solutions address these needs specifically. To address these challenges AWS team has released a solution called Serverless Transit Network Orchestrator (STNO). The STNO solution adds automation to the AWS Transit Gateway service. It provides the tools necessary to automate the process of setting up and managing transit networks on AWS environments using multiple VPCs. However, the STNO solution adds a lot of management overhead and may not be suitable for large enterprises. See details at AWS STNO. AWS is aware of these scaling and management challenges. At Re:Invent 2021, they released AWS Cloud WAN, a managed solution for creating and managing highly scalable, global, private networks.
AWS Cloud WAN
At re:Invent 2021, AWS announced the preview release of a new global, centralized, managed networking service, AWS Cloud WAN. With AWS Cloud WAN, customers can build, manage, and monitor a global network that connects resources running across your cloud and on-premises environments. Clients can connect their AWS or on-premise data centers to the Cloud WAN in a simplified manner and keep the network securely connected and high-performing. AWS Cloud WAN is currently in preview across 10 regions and is expected to go GA in early 2022.
Components
- Global Network – A single, global, private network that acts as the high-level container for your network objects. It can contain both AWS Transit Gateways and other AWS Cloud WAN core networks. These can be seen as the console for managing the WAN.
- Core Network – The part of your global network managed by AWS. This includes regional connection points and attachments, such as VPCs, Transit Gateway Connects, VPN and Direct Connect Gateways.
- Core Network Policy – a single document written in a declarative JSON like format applied to your core network that captures your intent and deploys it for you. It defines segments, AWS Region routing, and how attachments should map to segments. With a core network policy, you can describe your intent for access control and traffic routing, and AWS Cloud WAN handles the configuration details.
- Network segments – Segments are a way to isolate areas of the Core Network. They are “dedicated routing domains”, which means that by default, only attachments within the same segment can communicate.
- Core Network Edges – The Regional connection point managed by AWS in each Region, as defined in the core network policy. Every attachment connects to a Core Network Edge.
- Attachments – Attachments are any connections or resources that you want to add to your core network. Supported attachments include VPCs, VPNs, Transit Gateway route table attachments, and Connect attachments.
- Route Tables – Route Tables in the original attachments such as VPCs, Transit Gateways, and VPNs must be updated to route traffic to the Core Network Edges.
How it works
We highly recommend deploying AWS Cloud WAN in the Centralized Networking Account. Resource Access Manager (RAM) configured properly will allow cross-account resources to be automatically added to the AWS Cloud WAN based on Tags. To create AWS Cloud WAN Global Network, perform the following steps:
- Navigate to AWS Console > Network Manager > Create a Global Network.
- It may take a few minutes for the Global Network to be available.
- Create a Core Network.
- Create a Core Network Policy to define the network segments, ASN range, AWS Regions (At least one region must be supplied), and Tags for segments. These tags will be picked up by CloudWatch EventBridge and resources will be automatically attached to respective Cloud Segments.
- Apply the Network Policy.
- You can add tags as defined in your Network Policy to Transit Gateways and VPCs to automatically create the network attachments.
- Create VPC routes to route global network traffic to the ARN of the Core Network Edges.
Transit Gateway Global Architecture and AWS WAN Comparison
The following table shows a comparison between Transit Gateway Architecture and AWS WAN. AWS WAN is a managed, global service that removes some of the complexity of managing a web of regional transit gateways and route tables while still providing a highly-available, secure, global network.
| Transit Gateway Architecture | AWS WAN |
| Managed by Customer | Managed by AWS |
| Requires Management of TGW Routing Tables | Routing managed using pre-defined rules in Core Network Policy |
| Regional Resource. Requires TGW Peering between regions and route propagation to create a global network | Global Network by default |
| DIY Automation. Requires a tool like STNO | Advanced Automation – Built In |
| Regional Management | Global Management Dashboard |
Network Manager
AWS Network Manager provides a centralized dashboard of physical locations, devices, and links. It helps visualize and provide an overview of your global networks. This view can be a crucial tool in the networking team’s arsenal. It provides the following dashboards:
Physical Location
Topology Graph
Topology Tree
Monitoring of Individual Resources
Benefits
- Single unified global point of control – You can configure your entire network from one network policy, AWS takes the intent from that policy, and makes it true across your network.
- More managed and opinionated service – Cloud WAN builds part of the network for you – you define AWS Regions in your core network policy, and AWS handles peering and dynamic routing between those Regions. Full mesh connectivity is configured in a few clicks in the AWS Management Console (or lines of JSON, if you prefer).
- Centralized dashboard, monitoring, and events – Cloud WAN uses the AWS Network Manager console to show physical and logical topologies, a central dashboard (that includes your own Transit Gateways), detailed monitoring, and a centralized event bus through Amazon EventBridge.
- Built-in automation for attachments – With Transit Gateway, you define your own routing, propagations, and how attachments map to route tables. Often, adding a new route table or attachment results in a dozen or more downstream changes. With Cloud WAN, you can use attachment tags to automatically map attachments to segments, where you define relationships between segments through policy.
End Results
AWS Cloud WAN is a managed wide area networking (WAN) service that makes it easy to build, manage, and monitor a global network that connects resources running across your cloud and on-premises environments. It provides a central dashboard for attaching the connections to reach your branch offices, data centers, and Amazon VPCs to the AWS global network in just a few clicks. You can use simple network policies to centrally configure and automate network management and security tasks and get a complete view of your global network.
Final Thoughts
In this Solution Spotlight, we define the core foundations and best practices of starting your enterprise AWS network journey. Following these practices and blueprints, your company should be set for a highly scalable and interconnected global AWS Network. In closing, we have two final points we’d like to make.
Security and Network Design go hand in hand! Insecure networks are the primary target for attackers to get into a system. Network design and security are mutually tied with each other. Any network design should be reviewed with the AWS Well-Architected Framework security lens. Implementing well-defined InfoSec review processes can prevent major security issues. Proper controls and monitoring of your cloud network should be implemented to detect deviations from established security policies.
AWS is constantly evolving and innovating, and so are network architecture best practices AWS is constantly releasing new services and features based on client inputs and general industry trends. As new services and features are released, network architectures and best practices also evolve to incorporate these new features. The network teams should keep up to date on the latest best practices and implement these improvements to the company network as possible. For example, prior to 2018, transit VPC connectivity was generally configured via complex EC2 based VPN configurations that were not easily scalable and required high maintenance efforts. Transit gateway was introduced in 2018 which truly simplifies inter-VPC communication and provides scalability, and high availability. However, AWS WAN was introduced in 2021 which provides even better.
Interested in learning more?
If you are looking to provide automation, consistency, predictability, and visibility to your software release process contact us today.
