Demonstrate an understanding of OCI load-balancing offerings and related resources

Summary

There are two (2) types of load balancers: Load Balancer service; and Network Load Balancer. The IP address is assigned to the load balancer listener.

Reference

• Infrastructure Servcies
• Networking

Load Balancing

A load balancer distributes requests across multiple servers from a single entry point. There are two (2) types of load balancers:

1. Load Balancer Service
• Works at layers 4 and 7 for the HTTP, HTTPS, and TCP traffic. Used for load balanicng applications and processes.
• Acts as a reverse proxy
• Can achieve up to 8Gbps per load balancer
• Supports backend autoscaling
• Can terminate SSL connections
• Can have a web application firewall
• One load balancer instance for free (limited to 10 Mbps)
2. Network Load Balancer
• Can load balance packet forwarding, network traffic, and application for TCP, UDP, ICMP, and IP traffic.
• Can preserve the client header information
• Capabale of scaling to multi-terabits per second
• Supports backend autoscaling
• Low-latency network integrated load balancer
• Network load balancers are Always Free tier

OCI Load Balancer: Shapes

The load balancer has a flexible shape for a range of bandwidth (10 - 8000 Mbps):

• Minimum bandwidth provides instant readiness for load
• Maximum bandwidth allows control of maximum cost

Content-Based Routing: Host Based

Host-based routing: Multiple websites using a single load balancer and a specified set of back-end servers.

Different back-end sets are routed to based on a combination of host name, port, and protocol. For example,

• http://host1.example.com:80 can be routed to one back-end set
• https://host2.example.com:443 can be routed to another back-end set

Content-Based Routing: Path Based

Path-based routing: Optimize resource utilization by routing to independent back-end sets based on URL paths

This is an extension of host based routing in which the URL is considered as well:

• http://host1.example.com:80/app can be routed to one back-end set
• http://host1.example.com:80/videos can be routed to another back-end set

Public Load Balancer

The public load balancer is associated with a public listener which has a public IP address and is reachable from the Internet. This is restricted to a single region. The load balancer configuration is:

• in a regional public subnet, or
• in a AD-specific subnet with a standby load balancer in another AD-specific subnet in the same region.

Private Load Balancer

Private load balancer is created with a private IP address. Primary and standby load balancers can be in the same subnet, but they require an extra private IP address.

Load Balancing Policies

Load balancing policy is one of:

Round Robin

Requests are routed to next server in back-end set. Good for short requests.

Least Connections

Requests are routed to the server in the back-end set with the least connections. Good for long running requests

IP Hash

Requests are routed based on the IP address of the client. Good for short requests from a large number of simultaneous clients

Load balancer policy decisions apply differently fo:

• TCP load balancer
• Cookie-bases session persistent HTTP request (sicky requests)
• Nonsticky HTTP requests

Health Check

• Health check:
• Confirms availability of back-end servers
• Continuously monitors back-end servers
• Configures TCP-level or HTTP-level
• Is activated for:
• Back end
• Back-end set
• Overall Load Balancer
• Configure your health check protocol to match your application or service.

Metrics

• The OCI Metrics feature relays data about the health, capacity, and performance of your cloud resources.
• Load Balancer metrics provide citical metrics to manage/monitor your local OCI balancer infrastructure:
• The metrics(~40) are broken down by Listener, Back-end set, and overall Load Balancer level.
• These metrics are statistics calculated from relevant data points as an ordered set of time-series data.

Web Application Acceleration

Web Application Acceleration: Overview

Anticipated Outcomes:

• Improves customer experience
• Reduces application latency and system load

Use Cases:

Scenario 1

• Improve application performance and decrease server load.
• Action: Leverage WAA caching.

Scenario 2

• Decrease network load and further reduce latency.
• Action: Use WAA compression.

Scenario 3

• Monitor and observe.
• Action: View request logs and metrics to confirm caching and compression.

Customer Workflow:

• Step 1 - Create a WAA policy with caching and compression and apply to load balancer.

Web Application Acceleration: Understanding

Works with Layer 7 http/https protocols only. This is a policy that is attached to OCI Load Balancers.

Caching

• Only HEAD and GET requests are cached
• Only reponse that return HTTP status 200 are cached
• Cached content might not stay current with content on the back-end servers until the cache expires or is purged
• Content is cached until it expires or is purged even if the file is removed from the back-end server
• Back-end servers can get a drastic increase in traffic when any of the following events occur:
• Multiple cache items expire at the same time
• The cache is purged
• The Web Application Acceleration service is disabled
• The maximum size of the cache is 100 MB. After this maximum size is reached, the Web Application Acceleration service removes those resources that have been accessed the fewest times until the maximum size is no longer exceeded.
• You cannot cache files over 100 MB

Network Load Balancer

• Provides automated traffic distribution from one entry point to multiple servers in a VCN
• Is an ideal load balancing solution for latency-sensitive workloads (i.e., VoIP and IoT)
• Optimized for long-running connections in the order of days or months (i.e., database or WebSocket applications)
• Supports public and private load balancers
• Load-balances connections based on Layer 3/Layer 4 (IP protocol) data
• TCP/UDP/ICMP traffic can be load balanced
• Is offered as a free service
• Load Balancing Policy: Tells the load balancer how to distribute incoming traffic to the back-end servers:
• 5-tuple Hash (source IP and port, destination IP and port, protocol)
• 5-tuple Hash (source IP, destination IP, protocol)
• 2-tuple Hash (source IP, destination IP)
• Key differentiators:
• Private or public load balancer (with public IP address
• Scales up or down with no bandwidth configuration
• Source and destination IP addresses, and port preservation
• For source and/or destination IP preservation to be enabled, the OCI NLB must be a private NLB.

Concepts

Back-end Server:

Application server responsible for generating content in response to the incoming TCP, UDP, or ICMP traffic

Backend Set:

Logical entity defined by a list of back-end servers (compute instances, IP addresses, or both), a load balancing policy, and a helath check policy

Load Balancing Policy:

Tells the load balancer how to distribute incoming traffic to the back-end servers

Session Persistence (session affinity):

The 5-Tuple Hash policy provides session affinity with a given TCP or UDP session, where packets in the same session are directed to the same back-end server behind the network load balncer. Use a 3-Tuple or 2-Tuple network load balancing policy to provide session affinity beyond the lifetime of a given session.

Health Checks:

A test to confirm the availability of back-end servers. Support HTTP-level, HTTPS-level, TCP-level, and UDP-level helath checks.

Listener:

An entity that checks for incoming traffic on the load balancer's IP address

Public Network Load Balancer

• Assigns a public IP address to the listener
• Is highly available in a region
• Requires public subnets (regional subnet)
• Needs one private IP address from the subnet
• Ensures high availability and accessibility even when one of the availability domains has on outage

Private Network Load Balancer: Use Case

Transparent NLB

• Private Network Load Balancer as Next Hop Route Target with VCN Transit Routing (bump in the wire):
• The network load balancer routes user traffic to the firewall instances hosted behind it in the Hub VCN using VCN route tables. This user traffic would otherwise flow from the source directly to the destination.
• In this mode, the network load balancer does not modify the client packet characteristics and preserves the client source and destination IP header information
• Source/destination header (IP, port) preservation must be enabled on the OCI Network Load Balancer

Load Balancing: Comparison